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ABSTRACT 



This document contains four evaluations of Project ALERT 
(Adult Literacy Enhanced & Redefined through Training) , a workplace literacy 
partnership of Wayne State University, the Detroit Public Schools, and 
several city organizations, unions, and manufacturers in the automobile 
industry that was formed to meet employees' job-specific basic skills 
application needs in manufacturing and service operations, through the 
development of whole language curriculum and training. The reports are 
"External Evaluator's Final Report" by Jorie W. Philippi and "Year Two 
Report, January 1997," "Report on Site Visits, March 1996," and "Formative 
Evaluation Report, August 1995" (Larry Mikulecky, Paul Lloyd) . Extensive 
onsite investigations and job analysis resulted in development of customized 
whole language, interactive multimedia, and learner-centered training. During 
the 3 -year project 78 courses in communications skills, mathematics, and 
job-specific skills were presented to 683 employees. Philippi's evaluation 
using a modified Context- Input-Process-Product (CIPP) model found that there 
was a good deal of consensus about program goals among the partners and the 
participants, with the only area of divergence being that some participants 
expected to acquire a broader range of skills than the courses addressed. The 
curriculum materials were determined to be very good, especially the 
multimedia CD-ROMs, although not as many workplace examples were used as 
intended. Deadlines for the project were met, and participants mastered 
course work. Although employees and supervisors reported transfer of training 
and improvement in work productivity, not enough data were collected to 
determine the validity of these reported outcomes. (KC) 
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Introduction 



Background - 

The Wayne State University College of Education and Labor Studies Center 
(WSU) in partnership with the Chrysler Detroit Axle Plant and UAW Local 961, the 
UAW-Chrysler National Training Center, the Davis Tool & Engineering Company and 
UAW Local 174, the City Management Corporation, and the Detroit Public Schools, was 
funded initially by a 36-month grant award from the U.S. Department of Education 
National Workplace Literacy Program to research, develop, deliver, and foster 
institutionalization of functionally contextual workplace literacy curricula and training 
programs to up-grade the basic skills and increase the productivity of manufacturing 
workers. The program operated as a national workplace literacy demonstration project 
from November, 1994, through May, 1998 including a seven-month no-cost extension. 
The purpose of the demonstration was to determine the effectiveness of the partners’ 
proposed workplace applications of basic skills training model. 

The need for this project grew from a recognition by WSU and its partners of the 
changing job requirements in the manufacturing community for both large and small 
operations, especially in the areas of basic literacy and English proficiency for its 
workers. The industry/service partners employ a total of approximately 2500 workers. 
Surveys conducted prior to the grant period (1988 and 1994) indicated that 20-25 percent 
of the Detroit area manufacturing workforce was functionally illiterate and that for many 
of the workers, English was not their native language. Each of the partnering companies 
and labor organizations expressed specific needs, as well. For Chrysler Detroit Axle 
Gear Division and UAW Local 961, there were plans to increase overall use of computer- 
numeric controlled machinery in production; for the City Disposal Systems division of 
City Management Corporation, there were needs for additional qualified refuse removal 
truck drivers and for increased worker communication and computer capabilities; for 
Davis Tool & Engineering Company and UAW Local 174, there were needs for skill 
upgrades to qualify front line workers as apprentices in skilled trades and for increased 
levels of mathematics to facilitate use of Statistical Process Control methods. 

Prior to applying for this U.S. Department of Education National Workplace 
Literacy Demonstration Project grant, the partners had begun discussions with WSU to 
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foster the sharing of information and to clearly define manufacturing/service employee 
needs and requests (see above) and agency responses. These careful explorations to 
define needs and possibilities resulted in the partnering of WSU with the 
industries/services listed above, along with the UAW-Chrysler National Training Center 
and Detroit Public Schools, to apply for federal grant monies for provision of customized 
workplace basic skills training programs. Representatives from each organization met 
with WSU staff to ensure that the customized programs would directly relate to the skills 
needed for their operations and would responded to the needs of the targeted incumbent 
worker participants. To this end, the education/manufacturing/service/labor organization 
partnership members were committed to gathering additional data for performing a 
“front-end analysis” in order to assess the applied basic skills needs of targeted 
employee-participants. They also determined program goals, scope of related content 
areas, length of sessions, schedules, recruitment, piloting, and implementation plans. 
This cooperative relationship continued throughout the funding cycle. 

The developers of the Project ALERT needs- assessment instruments and 
instructional materials, (i.e., the WSU and Detroit Public Schools faculty members that 
comprised a professional staff of curriculum consultants and professors with numerous 
advanced degrees and many years of experience in writing and teaching), then custom- 
designed, created, delivered, and monitored program pilots for more than a: dozen 
different workplace-related subjects. Complete participant assessment procedures and 
strong, whole language (functionally contextual) workplace applied basic skills training 
programs, along with traditional and commercially developed literacy training programs, 
were subsequently implemented and refined during the grant period. Wayne State 
University, as grant fiscal manager, contracted with Performance Plus Learning 
Consultants, Inc. (PPLC) to serve as a third-party evaluator after the conclusion of project 
operations. 

Purpose of the Evaluation : Wayne State University College of Education and Labor 
Studies Center has requested this third-party evaluation of their U.S. Department of 
Education National Workplace Literacy Demonstration Project to assess: 1.), the extent 
to which the project’s goals and objectives have been accomplished; and, 2.), the extent 
to which program development, implementation, expansion, and institutionalization 
proceeded as planned. Specifically, the evaluation objectives to be investigated, based on 
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the project purpose, goals and objectives published in the proposal for funding (. Proposal , 
March, 1994, pp. 16-17), were: 

Project purpose : To provide for an innovative institutionalized workplace literacy 
program that will increase the skills of employees and improve the productivity of 
the organization and the quality of its products. 

Goal 1 : To design develop, and implement innovative workplace literacy 
programs that are tailored to the organization, the skills and cultural background 
of participants. 

Objective 1 : To conduct a comprehensive pre-design analysis of jobs, 
employees, and organizations to gather data requisite to the design of effective 
literacy training. 

Objective 2 : To collaboratively design and develop workplace literacy 
programs that are responsive to the unique needs of the organizations, jobs, 
and employees. 

Objective 3 : To design and develop a range of culturally sensitive 
instructional methods and materials that are both technology and non- 
technology oriented. 

Objective 4 : To design resource-rich classrooms that support traditional and 
technology-based instruction. 

Objective 5 : To recruit participants, implement the site-specific programs, and 
provide for reinforcement and transfer of training to the workplace. 

Goal 2 : To conduct project evaluation and program impact research. 

Goal 3 : To disseminate the program, program products, and research findings. 

Goal 4 : To promote program institutionalization and diffusion. 
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Description of the Project to be Evaluated : The Project ALERT Workplace Literacy 
Project consisted of a workplace literacy training partnership formed between WSU, the 
Detroit Public Schools, City Management Corporation, the UAW-Chrysler National 
Training Center, Chrysler Detroit Axle and UAW Local 961, and Davis Tool & 
Engineering Company and UAW Local 174. According to the published description of 
the program, the design of the project was structured to meet employees’ job-specific 
basic skills application needs in manufacturing and service operations, through the 
development of whole language (functionally contextual) curriculum and training. 
Extensive on-site investigations and job analysis conducted by project staff resulted in the 
gathering of organizational training and job materials and scenarios, as well as 
observation, worker and supervisor interviews, and in-depth analysis of applied basic 
skills used in job task performance. This information was then used for development of a 
customized whole language, interactive multi-media, and learner-centered training 
programs and instructional models tailored to meet the various organizational needs. A 
brief description of the programs follows: 

The programs consisted of workplace literacy interventions at four locations in 
proximity to the three partnering organizations and to small businesses in the 
geographical vicinity of the federally-funded Detroit East Empowerment Zone.: on-site 
classrooms at Chrysler Detroit Axle, an on-site classroom at Davis Tool, a trailer at the 
City Disposal Department site for City Management Corporation, and space at the UAW- 
Chrysler Wayne Family and Community Learning Center. Initial efforts focused on 
consistent needs assessment procedures, which were developed, piloted at Davis Tool, 
then modified to fit the needs of the other locations. More than 75 interviews were 
conducted with hourly employees, union officials, supervisors and managers from 
participating organizations, as well as observations of shop floor operations. Based on 
these literacy task analyses conducted at each location at the onset of this grant, three 
types of training were designed/purchased and implemented at specific sites: whole 
language oriented, interactive multi-media, and learner-centered instruction. Courses 
were piloted, then delivered throughout the grant funding period as 1. 5-3.0 hour units of 
instruction, one or two times per week. A total of 78 course offerings were presented 
during project operations, to a total of 683 employee participants. 

The following courses were developed and delivered: 
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Interpersonal Communication and Problem Solving 
Apprenticeship Prep 
Technology for the Workplace 
Numbers at Work 

SPC-Prep (Statistical Process Control) 

Effective Communication 

Math for Machine Operators 

Manufacturing Process Flow (CD-ROM) 

Gear Talk (CD-ROM) 

Effective Communication on Computer 

New Reading Disk (CD-ROM) 

Purchased or licensed training included: 

Quality Fundamentals and Data Skills (PLATO) 



Workplace Basics from PLATO (PLATO via Internet) 



Reading Improvement (PLATO) 



Commercial Driver ’s License Prep (Pennsylvania State University) 

Staff-developed module print materials were desk-top published and laser printed 
with careful attention to uniformity of format, layout design, graphics, and high quality 
reproduction. Additionally, instructional CD-ROMs were created for use in Computer- 
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Based Training (CBT). Participant materials and instructor guidelines, which also 
contained customized pre- and post-tests, were provided; and revisions were made to 
each course or session, based on feedback received from instructors, union officials, 
managers, workers, and the first external evaluators. 
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Method 

Design : This external evaluation of the Workplace Literacy Demonstration Project of 
WSU and its partners employed a modified version of the Context- Input-Process-Product 
(CEPP) model, (Stufflebeam & Guba, 1971). The CEPP model has been used to evaluate 
numerous workplace literacy programs across the country for the past 10 years. It 
provides feedback for project stakeholders as data collected is analyzed to develop 
summative reports at the end of specified cycles of program operation. This method of 
evaluation was chosen by the evaluator as the most suitable tool for investigating the 
evaluation objectives, (see pg. 5), because it examines project effectiveness through 
structured analysis of the cohesiveness of project goals, components, and operations, 
independent from comparisons to outside standards or other programs. This premise 
works well within the emerging field of workplace literacy, as well as with demonstration 
projects, because of the uniqueness of each individual program that is developed. 

A 

The CEPP model was used to analyze: 

• Context , i.e., the shared program goals and philosophy of key 
personnel and participants; 

• Input , i.e., resources, including personnel, materials, time, and 
facilities; 

• Process, i.e., congruence of program development, operations, and 
observed instructional delivery with project goals and research on 
instructional effectiveness; and, 

• Product i.e., indicators of program effectiveness and potential for 
program expansion and/or replication. 

It is important to note that, due to geographical, staffing, and time line 
considerations, the on-site data collection and project monitoring was conducted by the 
WSU project staff and reported to the evaluator via telephone, written, or electronic 
communications from the Project Director. Forms and procedures for use in data 
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collection across sites were developed and revised by the WSU project staff and their 
partners. 



Participants : The participants in the project were 683 employees of the project 

industry/service partners. Demographic data were made available to the external 
evaluator for 329 participants at Chrysler Detroit Axle, Davis Tool, and City Disposal 
Systems. A brief description of the available composite average participant profile (n - 
329) is as follows: 



Composite Average Participant Profile 

Thirty-nine year old, African-American male US citizen with a high school diploma 
whose first language is English. 



More detailed demographic information about participants is displayed in Figure 1 
below. For additional detail, please refer to Appendices. 



Figure 1: Demographic Characteristics of Participants (n = 329) 



Age: 






Ethnicity: 






Gender: 




20-29 years old 


14.6% 




White 


33.7% 




Male 


82.1% 


30-39 years old 


33.7% 




Black 


58.4% 




Female 


17.9% 


40-49 years old 


33.1% 




Asian 


0.9% 








50-59 years old 


17.9% 




Indian 


..1.2% 








60-69 years old 


0.6% 




Other 


5.8% 








Bora in US? 






English spoken at home? 






Yes 92.7% 






Yes 


96.0% 








No 7.3% 






No 


4.0% 








Education Level: 
















No schooling 


1-5 yrs. 




6-8 yrs. 


9 yrs. 




10 yrs. 


11 yrs. 


0.9% 


2.7% 




1.8% 


4.3% 




4.3% 


4.6% 


12 yrs. HS 




GED 


Some 




College 


Trade 




or more diploma 




college 




degree 


school 




76.3% 58.7% 




12.8% 


30.1% 




14.6% 


17.6 
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Instruments - Data for this evaluation were gathered via pre-and post-program participant 
surveys and structured interviews with participants and supervisors/managers, trainers, 
administrators, and program personnel. Additionally, data were gathered from detailed 
analysis by the external evaluator of program documentation, instructional materials, and 
participants’ work (i.e., pre-and post-assessment records). 

Procedure - Following initial telephone conversations with the Project Director to 
establish evaluation objectives, the evaluator conducted the activities listed below. 

1. Telephone consultation with the Project Director, Dale Brandenburg, to 
discuss project goals, progress, evaluation activities, availability of data and 
detailed explanations of project operations. 

2. Meetings with Dale Brandenburg: Cleveland, OH, late winter, 1997; and 
Milwaukee, WI, late spring, 1997. 

3. Meeting with Irene Sinclair, the Project Coordinator, Cleveland, OH, late 
winter, 1997. 

4. Off-site analysis of materials and data collected from site. 

5. Final External Evaluation Report submitted to Project Director, August, 1998. 
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Results 



Project Context: 

To what extent are goals and philosophy of the project shared by key project 
personnel and participants? 



This section of the evaluation is a comparison of the project goals and 
priorities as reported in project descriptions and interviews with key project personnel, 
including: 



• project director; 

• business partner supervisors; 

• project coordinator; and, 

• participants. 

These viewpoints about project goals were analyzed for consensus and divergence, using 
the following probes as guidelines for data collection and interview questions: 

1 . How have the goals of program instruction been defined? 

2. Is there a clear written statement to which all participants, instructors, and key 
program personnel subscribe? 

3 . What beliefs about workplace literacy are promoted by the program? 

4. Are those beliefs documented and accepted by those who are a part of the 
program? 

5. Are those beliefs supported by current theory and research? 

6. Is there a clear statement of program objectives that delineates how instruction 
is to occur at different phases of the workplace literacy program? 

7. Is the workplace literacy program, as defined, compatible with the needs and 
characteristics of the participants and of the company it serves? 

1 . How have the goals of program instruction been defined? The published project 
goals and purposes are contained in the grant proposal (March, 1995), submitted to 
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the US Department of Education. They were developed cooperatively following 
communication between Wayne State University and its partners prior to applying for 
grant monies. Stated goals in the proposal were: 

Goal 1 : To design develop, and implement innovative workplace literacy 
programs that are tailored to the organization, the skills and cultural background 
of participants. 

Objective 1 : To conduct a comprehensive pre-design analysis of jobs, 
employees, and organizations to gather data requisite to the design of effective 
literacy training. 

Objective 2 : To collaboratively design and develop workplace literacy 
programs that are responsive to the unique needs of the organizations, jobs, 
and employees. 

Objective 3 : To design and develop a range of culturally sensitive 
instructional methods and materials that are both technology and non- 
technology oriented. 

Objective 4 : To design resource-rich classrooms that support traditional and 
technology-based instruction. 

Objective 5 : To recruit participants, implement the site-specific programs, and 
provide for reinforcement and transfer of training to the workplace. 

Goal 2 : To conduct project evaluation and program impact research. 

Goal 3 : To disseminate the program, program products, and research findings. 

Goal 4 : To promote program institutionalization and diffusion. 

2. Is there a clear written statement to which all participants, instructors, and key 
program personnel subscribe? The published goals of the project listed in the program 
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program proposal serve this function. Interviews with key program personnel 
evidenced that they subscribed to these goals. Interviews were conducted with Dale 
Brandenburg, the Project Director, in person and over the telephone (1997-1998), and 
with Irene Sinclair, the Project Coordinator, in person (1997). Both of these key 
members of the Wayne State University Project ALERT staff provided descriptions 
of Project ALERT and its goals and objectives which closely matched those 
contained in the proposal. Dr. Brandenburg’s comments provided evidence that 
project operations had been designed and executed for the specific purposes of 
achieving the project goals and objectives. Ms. Sinclair’s comments provided 
evidence that all daily project activities, including curriculum development and 
delivery, also were undertaken to support the project goals and objectives. Off-site 
review of data from 108 learner pre-program expectation surveys collected by the 
project staff indicated that 44% of Project ALERT participant learning goals 
specifically mentioned instructional topics contained in the courses for which they 
had enrolled. 

3. What beliefs about workplace literacy are promoted by the program? Based upon the 
published goal statements, documented project activities (i.e., needs assessment, 
curriculum development and delivery, recruitment, and data collection queries) and 
the responses of key project personnel, there is evidence that the following beliefs 
about workplace literacy were being promoted by the program: 

• program should meet needs identified by employer and employees 

• use of a functional context (or whole language) approach in development 
of instruction 

• outcomes measured as impact on job performance and identified job needs 

4. Are those beliefs documented and accepted by those who are a part of the 
program ? Based on documentation and discussion of program activities, there is 
evidence that the program beliefs listed under Question 3. above are accepted by 
those who are a part of the program staff. Additional data was collected 
throughout the program to determine participant beliefs and their acceptance of 
program beliefs. Eighty-seven percent of participant responses on the Learner 
Expectation Post-Program Survey reported that they had learned or almost learned 
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what they had expected to in the courses and 98% reported that they would 
recommend the program to coworkers or staff. On a scale of 1 to 5, with 5 being 
the highest, in post-program 6-month follow-up surveys, participants rated the 
courses on average as 3.63 and rated themselves as better workers on average as 
3.46. (Please see Product topic in this section for a more detailed description.) 

5. Are those beliefs supported by current theory and research? The program beliefs 
are supported by current cognitive psychology and learning theories and research, 
indicating that transfer of learning from training situations to job performance is 
greatest when training context most nearly matches that of actual job situations. 
This program uses the results of literacy task analyses (needs assessment) as the 
context in which targeted skills for instruction are embedded, thereby allowing 
participants to experience skill learning and practice in context, and thus promote 
greatest impact of training on subsequent performance of skills in job tasks and 
everyday life tasks. 

The program content and instructional objectives initially stemmed directly from 
information gathered from workers and supervisors at Axle, City, and Davis 
locations. Analysis of responses to those needs assessment/structured interview 
questions indicated that the instruction produced for use during the project was 
directly related to those skills desired for performance of critical job tasks. 

6. Is there a clear statement of program objectives that delineates how instruction is 

to occur at different phases of the workplace literacy program ? Program 

instructions delineating how instruction is to occur at different phases of the 
project were clearly outlined in the project proposal, with steps, activities, and 
objectives listed for each year of planned program operations. 

7. Is the workplace literacy program, as defined, compatible with the needs and 
characteristics of the participants and of the company it serves? The information 
gathered from company employees, union representatives, and supervisors at each 
site at the onset of project operations are reflected in the curriculum materials that 
were developed. Program shareholder reactions to the courses developed and 
implemented were very positive and indicate acceptance and approval of both the 
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content and format of instructional materials in correspondence with company, 
worker, and union program goals. 



Goals of training participants were collected in individual interviews during site 
visits and on pre-program and post-program surveys administered by project staff for 
each course conducted. On pre-program surveys, participants responded to the question, 
“Specifically, what would you like to improve ?” Selected data, provided to the external 
evaluator, are listed below. The responses include a range of participant expectations for 
the courses. For a complete list of post-course survey responses detailing what each 
respondent felt they actually gained from the training, please see the Product section of 
this report and the Appendices. 



Participant Goals for Project ALERT Courses 


Course 


Participants’ Learning Expectations 


SPCI 


Math, test taking 


SPCI 


All 


SPCI 


Everything 


ECC 


I would like to be computer literate. 


ECC 


Learn more about the computer. 


ECC 


Understand more about computers. 


ECC 


A general knowledge of computers. 


ECC 


Knowing how to buy a computer — keeping up with daughter who is interested in 
computers. 


ECC 


Math, reading, and computer skills. 


ECC 


How to turn it on. 


ECC 


Just learn 


ECC 


Learn how to work the computer-operating Windows and write papers on the computer 


ECC 


Completely computer literate 


ECC 


Reading and computers 


ECC 


Completely computer literate 


ECC 


Computer skills and the basic functions. 


ECC 


Computer knowledge 


ECC 


My understanding about computers 


ECC 


My ability to communicate with others. 


ECC ~ 1 


Formatting spread sheets, editing, and making files. 


ECC 


Math — out of school too long 


ECC 


Learning to set up my own programs. 


ECC 


I’d like to know more about the computer because I have one at home. 


ECC 


There is always room for more learning, so anything and everything. 
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ECC 


Any skills or anything that can improve myself as a person and with other people in my 
other education areas. 


ECC 


Basic knowledge or functions of the computer 


ECC 


Having basic knowledge of computers so I can get advanced training. 


ECC 


Story problems, algebra, business math, and computer functions 


ECC 


Spelling skills and building my vocabulary 


ECC 


Learning major programs, how to upgrade, and faxing with computers 


ECC 


Improvement in the advantage computer skills and reading and writing 


ECC 


Knowledge of computers 


ECC 


Learn the basic use of the computer. 


ECC 


Learn basic computer skills and learn to use the Internet 


ECC 


Access library information from the computers; basic computer skills 


ECC 


Knowledge of computers, overall usage 


ECC 


Computers and math skills 


ECC 


Knowledge and skill of computers 


ECC 


Computer skills 


ECC 


Understanding of computers 


ECC 


Improve basic functions of a computer 


ECC 


Operating a computer, including finding information and maneuvering 


ECC 


To learn the ins and outs of a computer 


ECC 


Knowledge and ability to comprehend and operate phases of the computer 


ECC 


Computer skills 


IPC 


All of the above skills 


IPC 


I would like to improve my communication skills 


IPC 


I would like to learn to work with computers 


IPC 


I would like to improve my algebra and math skills and computer skills 


IPC 


Basic computer knowledge 


IPC 


Math improvement 


IPC 


Communication 


IPC 


Spread sheets 


IPC 


Math, problem solving, interacting with others 


IPC 


Communication skills with coworkers 


IPC 


Communication skills, reading/ writing improvement 


IPC 


Computer literacy, communication skills 


IPC 


Computer skills, math improvement 


IPC 


Recognizing and accepting changes 


IPC 


Basic and advanced knowledge of skills in computers and design 


IPC 


Math 


IPC 


Learn more about how classmates resolve their problems 


IPC 


Communication skills 


IPC 


Math computations, basic computer skills, and writing improvement 


IPC 


Improve myself 


IPC 


Public speaking with large groups and ways to better resolve problems in a professional 
manner. 


IPC 


Relate job ideas and concepts to employer and manager 


IPC 


My math skills and test taking skills 


IPC 


Upon completion of this course, I’d like to become a better listener. 


IPC 


Listening better and expressing myself better 
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IPC 


Learn how to communicate with people who I have a difference with and that I don’t 


IPC 


Communication skills, and computer skills 


IPC 


To listen clearly and communicate better. 


IPC 


I would like to communicate with people better. 


IPC 


How to deal with snobbish people — those who think they are better than others 


IPC 


Being better at communicating by talking and listening 


IPC 


My computer and algebra skills 


IPC 


I would like to complete my Associates in Applied Science in Information Systems 


IPC 


Computer skills 


IPC 


Communication and writing skills 


IPC 


Reading and writing 


IPC 


Basic overall improvement 


IPC 


Better ways of communicating with others 


IPC 


How to communicate with a person that is very upset — what, how, and when to say 
something. 


IPC 


To be more considerate of others and their opinions 


IPC 


Computer skills because I don’t have any 


PreCNC 


In college, I did not do well in calculus 


PreCNC 


Understanding of CNC programs and their functions 


PreCNC 


Computer skills 


PreCNC 


Basic skills and orientation in computer language 


PreCNC 


Math is where I most need to improve 


PreCNC 


Reading and writing 


PreCNC 


Identifying angles 


PreCNC 


Math 


PreCNC 


Become a better speller so that I can utilize my computer skills 


PreCNC 


Problem solving with computer, math basics, and feeling more comfortable with 
computers 


PreCNC 


CNC programming 


PreCNC 


Math and computers 


PreCNC 


Would like to be more computer literate 


PreCNC 


Math, reading, and computer skills 


PreCNC 


Math 


CNC 


Math, computers 


CNC 


Computer usage, reading and writing 


CNC 


All of the above 


CNC 


Test taking skills 


CNC 


Computer and communication skills 


CNC 


Math, reading, writing, computer, and communication skills 


CNC 


Algebra, geometry, percents, and decimals 


CNC 


I need more computer knowledge. 



PPLC analyzed goal statements from the project director, project coordinator, 
supervisors, and employee participants. For a discussion of areas of convergence and 
divergence, please see the evaluation section, “Summary of Results,” under Discussion . 
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PPLC next investigated the input of resources to the project, which is addressed in the 
next section of the evaluation. 
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Project Input : 



What resources were available to the project during development and 
implementation and to what extent were they used effectively ? 

This section of the evaluation addresses major resources of the project. It 
includes program instructional materials, design and appropriateness for the targeted 
participant population; key personnel qualifications and the match between published 
project duties; and facilities. It also examines the content and processes used for 
instructor training. The data presented in this section were analyzed for strengths and 
weaknesses, using the following probes to guide the investigation interview questions and 
data collection: 

1 . Are the workplace literacy program materials consistent with its stated philosophy 
and goals? 

2. Are adequate materials available for all phases of the workplace literacy program? 

3. Do the workplace literacy materials and instructional techniques accommodate 
the literacy levels of the program participants? 

4. Are instructors adequately trained to implement all phases of the workplace 
literacy program? 

5. Are effective support services readily available to participants who need them? 

6. Is the learning facility planned and equipped to support the workplace literacy 
program? 

7. Is the workplace literacy program record-keeping system complete, simple, and 
efficient? 

8. Are the expertise, training, and experience of key program personnel being 
utilized appropriately in the development and implementation of program 
activities for which they have responsibility? 



1 . Are the workplace literacy program materials consistent with its stated philosophy 
and goals? The instructional materials developed by project staff were based upon 
the results of the needs assessment process conducted with partnering organization. 
Additional commercially developed materials integrated into instruction were also 
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selected based upon the results of these needs assessments (i.e., the Commercial 
Driver’s License course for City workers). Two courses, considered representative of 
the stand-up training and computer-based training materials developed by the Project 
ALERT staff, were made available to the external evaluator for review. The review 
resulted in the following: 

SPC-Prep - Instructor’s Guide - a spiral bound paperback, consisting of 147 
pages, printed single-side. Introductory materials (22 pages) include - 

• Course overview 

• Rationale for curriculum approach 

• Implementation information, containing information on the needs of 
adult learners, setting the classroom learning environment, suggestions 
for working with various learning styles, and delivery tips 

• Lexicon of terms describing various educational activities 

• Instructor role and responsibilities 

• Course objectives 

• Description of session components and formatting 

• Planning and scheduling for instructional unit delivery 

The course is divided into two sections, SPC-Prep 1 (80 pages) and SPC-Prep 2 
(34 pages) and includes “previews” or pretests, which also serve as posttests, plus 
scoring guides (without answers), for each part. The first part of the course is 
designed to be delivered in 24 hours of instruction; the second part, in 12-15 
hours. No specific time is suggested for delivery of any one instructional unit. 

Materials for each session include a formatted 1-2 page set of directions for 
learning activities, followed by copies of participant materials for the specific 
session. The directions for each session contain a statement of the focus of the 
session and/or its desired outcome; a list of materials required for delivery of 
instruction; suggested activities to activate learners’ prior knowledge of the topic; 
description of an exercise or exercises focusing on specific computational or math 
awareness skills; a brief statement of how the skill relates to workplace tasks or to 
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everyday life activities; and, a space for written instructor evaluation of the 
session success. 

Topics for instruction are- 

“SPC-Prep 1”- 

• Math Memories 

• Keeping Track of Learning 

• Introduction to Math Operations 

• Place Value 

• Whole Number Practice 

• Round ing Off Number s 

• Finding the Average or Mean 

• Introduction to Decimals 

• Math Operations with Decimals 

• Basic Operations Using Decimals 

• Rounding Decimals 

• Positive and Negative Numbers — Signed Numbers 

• Number Lines 

• Rules for Working with Positive and Negative Numbers 

• Addition and Subtraction with Positive and Negative Numbers 

• Rule for Adding 3 or More Signed Numbers 

• Multiplication and Division Rules with Signed Numbers 

• Introduction to Fractions 

• Basic Operations with Fractions 

• Common Denominators 

• Basic Operations with Mixed Fractions 

• Changing Fractions to Decimals 

• Changing Fractions to Decimals and Percents 

• Finding the Percentage of a Whole Number 

“SPC-Prep 2”- 

• Keeping T rack of Learning 

• The Need for Quality 

• Changes in the Approach to Quality through History 
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• Quality in the Workplace 

• Introduction to Statistics 

• Statistical Terms 

• What is Statistical Process Control? 

• Recognize, Use and Understand SPC-Related Terminology 

• Introduction to Tables, Charts, and Graphs 

• Types of Graphs 

• Acceptable Range 

• QW 9000: What Can Customers Expect from a Supplier’s Quality 
System? 

None of the sessions contained actual workplace materials or scenarios from the 
partnering organization(s) for illustration or practice of skills addressed during 
instruction. The only mention of the workplace was in word problems framing 
operations with mixed fractions, and these were either unrealistic or imagined, rather 
than actual workplace uses of the skills. For example, 

“The factory needed new security fencing around the plant. The land around the 
factory measured 3 7/10 miles on each of its four sides. How many miles of 
security fencing is needed to replace the old fencing? ” [It is highly unlikely that 
security fencing would not be installed by an outside firm, rather than class 
employee participants.] 

“If an oil drip pan weighed 10 Vi pounds, how many could you cut from 840 
pounds of steel ? (Do not he concerned about scrap pieces.)" [It is highly 
unlikely that a worker responsible for cutting or stamping parts from a sheet of 
steel would be concerned with weight, rather than with sheet dimensions, part 
dimensions, kerf dimensions from cuts, AND with layout resulting in the least 
amount of scrap.] 

A word list for SPC-Prep 1 terms, a glossary for SPC, and a bibliography of sources, 
from which some of the instructional materials were adapted, were also included in 
the Instructor ’s Guide. 
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SPC-Prep 1 -Participant ’s Manual - a paper, pocket-folder binding 60 pages of 
participant materials for use during instruction. In addition to topic-specific 
worksheets for practice exercises, the manual contains: 

• Introduction to the course 

• Course goals [instructional objectives] 

• Directions for using the Participant ’s Manual 

• Form for self-evaluation of learning to be used with each session (1 
copy) 

Math for Machine Operators: GET IT IN GEAR / -Instructor ’s Guide- spiral bound 
paperback, consisting of 131 pages, printed single-side. Introductory section (36 
pages) contains: 

• Overview of course purpose, objectives, and time requirements 

• Course description 

• Scope and sequence chart for units of instruction 

• Suggestions for interacting with participants 

• Suggestions for using assessment to individualize instruction 

• Techniques (for participants) to use for studying mathematics 

• Decision-making guidelines for use of calculators by participants 

• Implementation information, containing information on the needs of 
adult learners, setting the classroom learning environment, suggestions 
for working with various learning styles, and delivery tips 

• Lexicon of terms describing various educational activities 

• Instructor role and responsibilities 

• Course objectives 

• Description of session components and formatting 

• Planning and scheduling for instructional unit delivery 

• List of recommended resources for supplemental materials 

The course utilizes two instructional methods — stand-up training with 
paper/pencil materials and computer multi-media instruction accessed from a CD- 
ROM. It is designed for integrated instructor-led delivery within 40 hours, (two 
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2-hour sessions for 10 weeks), but also has the flexibility of being used as 
independent self-instruction by participants. The Instructor’s Guide includes 
“previews” or pretests, which also serve as posttests, plus an answer key. 

Materials for each session include a formatted 2-4 page set of directions for 
learning activities, followed by copies of participant materials for the specific 
session. The directions for each session contain a unit and session title, 
description of session goal and objectives, a list of prerequisite skills; reference to 
the portion of the CD to be used for the session; a list of materials required for 
delivery of instruction; suggestions for delivery of instruction; reference to a 
commercially published topic-specific general math pretest/posttest source; a 
statement of the targeted participants’ purpose for learning the topic; description 
of paper/pencil and computer exercises focusing on specific math skills; 
suggestions for evaluating participant learning; commercially published resource 
materials; and an exercise or suggested exercise(s) in commercially published 
math workbooks for homework assignments. 

Topics for instruction are- 



• Whole numbers 

• Place Value, Rounding, Addition, Subtraction 

• Multiplying and Dividing 

• Multiplication Tables 

• Exponents and Square Roots, Order of Operation 

• Decimal Numbers 

• Decimal Place Value 

• Reading and Writing Decimal Numbers 

• Rounding Decimal Numbers 

• Adding and Subtracting Decimal Numbers 

• Multiplying Decimals 

• Dividing Decimal Numbers 

• Positive and Negative Numbers 

• Adding and Subtracting Positive and Negative Numbers 

• Multiplying and Dividing Positive and Negative Numbers 
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• Percents 

• Converting Decimals and Percents, Percent Calculations 

• Measurement and Statistics 

• Average (Mean), Mode, Median, and Range 

• Geometry 

• Angles 

• Circles 

• Cartesian Coordinate System 

• Metric System 

• Magnitude 

By design, the course was generic in its content, using only one workplace example in 
the paper/pencil materials, a Sample Hofler Report for CNC equipment displaying 
machine corrections to be entered into a computer. The general approach to math 
used by machine operators was chosen to provide greater flexibility for use of the 
course in a variety of small to medium manufacturing environments {Instructor’s 
Guide, pg. 7). 

A glossary for math for machine operators was also included in the Instructor’s 
Guide, along with a list of references used in developing the course contents. 

Math for Machine Operators: GET IT IN GEAR! - CD-ROM for Participants- 
An IBM-compatible CD-ROM, operating in a Windows 95 environment, minimally 
requiring 16 MB of ram, a 256-color monitor set to 600x800 resolution, a mouse, a 
keyboard, a 4x CD-ROM drive, and a 486 processor with sound card and speakers. 

The CD-ROM manual lists 9 lessons: 

• Decimals and Integers 

• Adding Decimals 

• Subtracting Decimals 

• Adding and Subtracting Positive and Negative Decimals 

• Understanding Averages and Ranges 

• Reading and Writing Decimals 
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• Understanding the Metric System 

• Understanding Magnitude of Small vs. Large Numbers 

• Understanding the Cartesian Coordinate System 

Additionally, during review, the external evaluator determined that the CD-ROM 
contains a tenth session, entitled “Combining Integers.” By design, the graphics are 
generic, {Instructor’s Guide, page 7), rather than workplace specific, although 

a 

approximately one third of the graphics are workplace-related. Video clips are 
utilized in some sessions as visual vehicles for relating session contents to users’ prior 
experience. One session is completely workplace related, using a CNC machine 
computer screen displaying a Hofler Report into which the user is required to enter 
calculated machine corrections. Lessons contain 20-25 screens with links to 
additional support screens. Average readability of the text on those screens selected 
for sampling by the external evaluator is an estimated reading grade level of 9.9 
(FORCAST formula for workplace materials. Ford, Fox, Caylor, and Sticht, 1971). 
Sentence length on the screens sampled ranged from 7 to 3 1 words, with an average 
sentence word count of 1 3 (screens in Understanding Magnitude of Small vs. Large 
Numbers lesson). 

The program is mouse driven, with a bank of icon buttons at the bottom of each 
screen. For example, “Next” moves the program forward to the next screen; 
“Details” provides additional information about the math topic or example displayed 
on the screen. When buttons are inoperable for specific screens, they are greyed out. 
For a user with previous experience in the Windows environment, the use of the 
buttons is intuitive. Each screen contains a “Narrator” icon button depicting a head 
set which activates an audio version of the text on the screen. This feature not only 
accommodates users with auditory learning style preferences, but also facilitates use 
of the program by readers whose ability level is not as high as that used in the screen 
text displays. All screens are designed to appeal to adult users. Examples of math 
use are interesting and informative in content and in the accompanying graphics and 
animation accompanying the text. 

Disclaimers in the licensing agreement reference the fact that the developer/copyright 
holder, Wayne State University, cannot be held liable for errors or inaccuracies 
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contained in the CD-ROM {GET IT IN GEAR! CD-ROM Manual, page 6). Several 
“bugs” were encountered by the external evaluator while reviewing the CD contents! 
These included: 

• inability to exit the lesson by clicking on the “Quit” icon button at the end of 
the test questions at the end of the lesson, or by the “Quit” icon button in the 
main menu, or by using the “Escape” key. 

• missing narration in several screens 

• error messages appearing on screens occasionally 

• “Details” screens indexed to incorrect main screens ( e.g ., “Details” screen for 
screen #20 activated by “Details” icon button on screen # 1 in Magnitude of 
Numbers lesson) 

Lessons conclude with several practice questions followed by several test questions 
whose responses are recorded in the program management system. The question 
formats include true-false, short answer, and short essay. The questions relate to key 
concepts presented in the lesson. Entry of responses is keyboard skill dependent for 
short answer and short essay formats. Programming denoting a response as correct 
appears to be limited to exact responses. Question stems contain syntax that is 
sometimes incompatible with the recognized response. For example, the stem of one 
short answer question required the response “is large” but the program counted the 
answer as incorrect if more than just the word “large” was entered as a response. 
Some test questions require an instructor to review the user’s entry on the screen and 
judge the correctness of the response. If no instructor is available, the program may 
be advanced by depressing the ENTER key twice, but there are no directions to the 
user indicating that this action is necessary. 

2. Are adequate materials available for all phases of the workplace literacy 
program? Materials were developed for the Davis site following extensive 
analysis of job tasks as part of the needs assessment process. Following pilot 
phases, curriculum was refined and also customized to meet the needs of the City 
and Axle sites, and additional courses were developed to complement the 
originally developed courses. There were no reports of inadequate instructional 
materials during any phase of project operations. 
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3. Do the workplace literacy materials and instructional techniques accommodate 
the literacy levels of the program participants? Studies done in the Detroit area 
prior to the partners applying for this grant (1988, 1994) indicated that the target 
audience for grant services were likely to have a 25% rate of functional illiteracy, 
a high incidence of first languages other than English, and low levels of formal 
education. The available demographic data collected by Project ALERT (n = 
329) showed that the majority of participants were bom in the United States 
(92.7%), speak English at home (96%), and have completed 12 or more years of 
formal education (76.3%). Data also showed that almost one third of the 
participants had some college. Test of Adult Basic Education (TABE) tests, 
administered to diagnose individual needs and estimated grade levels of ability in 
reading and math, were reported as raw scores rather than as scaled score grade 
equivalents to protect the privacy of participants. For this reason, it was not 
possible to use these scores as a means for determining the general compatibility 
of participant literacy levels with materials written at an estimated 9.0-10.0 
reading grade level and addressing estimated 5.0-10.0 grade level math concepts. 
However, research indicates that the average high school graduate in the United 
States in the mid 1970s could read at an estimated reading grade level of 8. 0-9.0 
and perform math skills at an estimated 6. 0-8.0 grade level. Since the mean age 
of project participants is 39 and the majority are high school graduates (58.7%), it 
can be assumed that the materials accommodated the literacy levels of most of the 
targeted participants. Additionally, materials reviewed by the external evaluator 
contained previews that further diagnosed specific needs of learners before 
instruction was delivered. Instructional units began with review or re-teaching of 
basic concepts. The CD-ROM includes narration of text on screens, which 
provides additional accommodation for individual participants with low reading 
or English language ability levels. In addition to formal instructional sessions, 
instructors at the Davis and City sites scheduled extra tutorial sessions to help 
employees with special needs. 

4. Are instructors adequately trained to implement all phases of the workplace 
literacy program? The Project Director reported that the Program Coordinator 
hired and conducted training sessions for all of the contracted project instructors. 
The external evaluator was not provided with content of the training sessions; 
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however, the Instructor ’s Guides that were reviewed contained extensive 
information on suggested delivery techniques and comprehensive outlines of 
session content. In addition to the pre-service training and monitoring of 
instructors conducted by the Program Coordinator, a 12-session staff development 
seminar was conducted from August, 1996 to May, 1997. Evidence of successful 
training also can be inferred from the high levels of participant satisfaction and 
mastery of course objectives and the thoroughness with which data were collected 
by instructors during project operations. (Please see the “Product” section of this 
report for additional detail.) 

5. Are effective support services available to participants who need them ? Support 
services were available to participants on several levels. Union members had 
access to the many social and educational services provided by their 
organizations. WSU conducted regular meetings with joint management and 
union representatives throughout the project, which provided opportunity for 
discussion and resolution of any necessary support services for participants 
beyond those built into the courses. The comprehensive educational background 
of the Program Coordinator, an employee of the Detroit Public Schools who was 
directly responsible for the training and monitoring of instructors, included 
extensive experience in adult education and upper level coordination of 
educational services and social agencies. This background provided an additional 
avenue for identification and acquisition of appropriate support services for 
participants beyond those inherent in the courses. Another level of support was 
built into the project structure as a model for supervisor orientation and supervisor 
follow-up for instruction. The purpose of this project component was to ensure 
that “front-line supervisors understood what employees would learn in the classes 
and how they could support the new learning behavior” (1998, Final Performance 
Report, page 15). Yet another level of support services was evidenced in the 
provision of facilities at the UAW-Chrysler Wayne Family and Community 
Learning Center and of 20 new multi-media computers by the UAW-Chrysler 
National Training Center; in the contribution of funds to construct a classroom at 
the Axle plant site; in the renovation of a storage area for on-site classroom use by 
Davis Tool; and in the refurbishing of a trailer with new heating/air-conditioning 
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systems, phone lines, furniture, and computers by City Management at the City 
Disposal site (1998, Final Performance Report, page 4). 

6. Is the learning facility planned and equipped to support the workplace literacy 
program? As stated above in Question 5., facilities were constructed, renovated, 
equipped, and/or rearranged for use for instructional delivery and project 
meetings by each of the partners. Instructional areas at all four sites (Axle, City, 
Davis, and Wayne Center) were designed to support both stand-up training and 
computer-based training. The Project Director reported that employees suggested 
classroom location preference and design at Davis; and, at Axle, large, bright, 
well-furnished and well-equipped classroom areas with computers were made 
available for use by the project. The external evaluators originally hired by the 
project, Drs. Larry Mikulecky and Paul Lloyd of Indiana University, visited three 
of the sites in March, 1996 and submitted the following information on facilities 
in their site reports (May, 1996): 

• Davis Tool -‘The classroom, which is located above the shop floor, is 
a former storeroom that has been converted by the company into a 
teaching area. All interviewees agreed that the classroom was very 
suitable, the instructor and one learner comparing it favorably with 
school classrooms they knew. Its location right on site was seen as a 
great advantage. However, this also produced its one major 
disadvantage — the noise and vibration from a press operating 
underneath the classroom. Only one possible change to the classroom 
was mentioned: the instructor would like more board space. 
[Objective 1.4 To design resource-rich classrooms that support 
traditional and technology-based instruction. The classroom provided 
by Davis is adequate to its purpose. The only qualifications to this 
concerned the noise level and a desire for more board space. If the 
program expands to include more emphasis on individual learner 
development, then more learner materials and space for their display 
would be needed.” (pp. 3, 5) 
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• City Disposal -“The classroom is a trailer which the company has 
brought to the worksite. This location on site was seen by all 
interviewees as a great advantage: ‘There’s no temptation to drive off 
home instead of to class.’ However, the small size of the trailer is a 
problem (‘The fellas are kinda big’) and has become more so as 
demand for classes has increased. The General Manager commented: 
‘If I’d known. I’d have gotten a bigger trailer.’ The solution at present 
is to have multiple classes — and waiting lists. In addition, one of the 
participants mentioned the inconvenience of having no bathroom close 
to the trailer. The instructor was impressed by the response to any 
requests that she had about locks or electricity: ‘If ever I need 
anything, they’re right here.’ f Objective 1,4 - To design resource-rich 
classrooms that support traditional and technology-based instruction. 
The trailer provided by City Disposal is adequate to its purpose. The 
only qualifications to this concerned its small size and the lack of a 
bathroom.]” (pp. 3, 5) 

• Chrysler Detroit Axle - ‘The Production Manager had high praise for 
the help that the project had given them in designing the new 
classrooms, ‘which we use a lot.’ One contained 12 computer stations 
and the others were fitted up well for class or small group teaching. 
[Object 1.4 - To design resource-rich classrooms that support 
traditional and technology-based instruction. The multimedia and 
other classrooms provided by Chrysler Detroit Axle are of high quality 
and very suitable for their purpose.” (pp. 2, 4) 

7. Is the workplace literacy program record-keeping system complete, simple, and 
efficient? Data were gathered for two different project activities, during the 
instructional development pre-design analysis phase of operations and during and 
after implementation to conduct impact research and evaluation tasks. 
Documentation of pre-design analysis was facilitated by three instruments that 
were designed, piloted, revised, and validated by project staff: Hourly Employee 
Interview , Supervisor Interview, and Observation Checklist. These instruments 
were used to standardize data collection for curricula design across sites. Data 
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were collected with the instruments by interviewing approximately 55 hourly 
employees, union officials, and supervisors, and by observing shop-floor 
operations for pertinent job tasks. Instructors collected data for impact research 
and evaluation. Data collection instruments used are detailed below: 

• TABE- Reading Test- Four reading passages ranging from 350-415 
words in length. Passages are in different formats (recipe, classified 
section of telephone directory, informational articles), each followed 
by five to seven multiple choice reading comprehension questions. 
Timed test, 25 minutes to complete 25 questions. Administered before 
and after each course, except during last 6 months of project 
operations in 1997. 

• TABE- Math Test- Fifteen problems requiring use of addition, 
subtraction, multiplication, and division of whole numbers; mixed 
fractions,; division of fractions; calculations with decimals, percents, 
and negative numbers. Timed test, 9 minutes. Administered before 
and after each course, except during last 6 months of project 
operations in 1997. 

• Learner Enrollment - Forty items, including demographic information 
and self-ratings of job literacy abilities, workplace context, and 
perceptions of basic skills. Administered during participant intake 
process. 

• Learner Assessment —Twenty items, including self-assessments of 
abilities and work environment, interest in additional classes, and 
possible impact factors that may have occurred since the course began. 
Administered at the completion of each course. 

• Learner Expectation - Open-ended questions for indicating areas in 
which participants hoped to improve while taking the course and areas 
in which participants perceived they had improved after taking the 
course. Top half of form administered prior to instruction, bottom half 
of form completed after completion of instruction. 

• Project ALER T Participant Survey - Four open-ended questions 
asking what participants liked, disliked, and thought were most 
important about the course, plus open comments. 
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• Six-Month Survey and -Twelve-Month Survey - Two part instruments, 
similar to each other. First part: 39 objective, 5-point scaled questions 
concerning course outcome impact on the job and in everyday life. 
Second part: 13 questions for a 20-30 minute person-to-person 
interview. 

• Customized Pre/Post Course Mastery Tests- Designed to reflect 
content of specific courses. Number of items varied by course. For 
the two courses reviewed by the external evaluator, pre/post course 
mastery tests were identical instruments, containing 8 to 19 items. 
Untimed. 

8. Are the expertise, training, and experience of key program personnel being 
utilized appropriately in the development and implementation of program 
activities for which they have responsibility? The Project Director provided a 
detailed list of proejct duties for each of the key project members, including the 
Project Director, the Assistant Director for Instruction, the Program Coordinator, 
the Project Secretary, the Co-Principal Investigators, the Graduate Research 
Assistants, the DPS Liaison, the Student Assistants, the Accounting Assistant, the 
UAW-Chrysler Liaisons, the Instructors, and the Video Technician. These duties 
were matched to those WSU professional record forms, vitae, and resumes made 
available to the external evaluator. Many of the project staff held advanced 
degrees. Of the paired information able to be obtained, there was evidence that 
the project responsibilities of most of the key personnel closely matched their 
educational specialties and experience. Information published in the Final 
Performance Report (1998) provided evidence that most of the duties assigned to 
various areas of responsibility during project operations had been completed or 
were scheduled for completion in the near future. (Please see the “Process” 
section of this report for more details.) 

For a discussion of strengths and weaknesses of available project resources and the 
effectiveness of their use, see "Summary of Results" under Discussion section of the 
evaluation. The next section of this evaluation examines the process of project delivery. 
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Project Process : 

To what extent were program operations, development and observed 
instruction congruent with project goals and research on instructional 

effectiveness? 

This portion of the evaluation report examines program operations, development, 
observed instruction, and daily activities to determine their level of congruence with 
project goals and current research on effective workplace literacy, using the following 
probes to guide interview questions, data collection, and data analyses: 

1. Are current records of routine program activities in the workplace literacy 
program maintained and do these activities reflect stated goals? 

2. What are each participant's current progress, instructional activities, and learning 
needs? 

3. Are the workplace literacy program instructional decisions and activities 
generally consistent among instructors who have similar responsibilities or who 
serve the needs of similar participants? 

4. Are participants in the workplace literacy program making the progress that was 
anticipated? How is this determined? 

5. How much time is spent in instruction with workplace literacy program 
participants in whole group, small group, individual formats? 

6. Are the workplace literacy resources planned for use actually being used? 

7. Is there a need for additional resources not initially planned for? 

1. Are current records of routine program activities in the workplace literacy 
program maintained and do these activities reflect stated goals? The extensive 
database developed by the project contained thorough records from all classes, 
including course completion rates, pre/post test scores, and participant 
assessments of course applications to their jobs and to everyday life activities. 
Seventy-eight classes were delivered in addition to the computer learning courses 
delivered at the Wayne Center. Summaries of project pre/post learning gains 
displaying mean scores evidenced gains for all courses at Davis, City, and Axle 
sites, indicating mastery of skills and concepts by many employee participants. 
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Reports prepared by the previous external evaluators, Mikulecky and Lloyd in 
1996 and 1997 indicated consistent progress on development of traditional and 
technology-based curricula for use in project operations, successful recruitment 
and development of instructional facilities, and somewhat less than adequate 
progress in analysis of collected data for impact research and cost/benefit 
analysis. Annual performance reports prepared by the Project Director contained 
documentation of project activities conducted during each year of project 
operations, categorized by project goals and deliverables. A matrix contained in 
the Final Performance Report (1998) details progress on curriculum 
development, implementation activities, program expansion by sites, research, 
institutionalization efforts, dissemination, and delivery of instruction. 

2. What are each participant’s current progress, instructional activities, and 
learning needs? Pre-instructional testing with customized course mastery 
instruments and the TABE, along with data collected from the Learner 
Assessment and Six-Month Surveys document participant progress toward 
individual learning goals. Combined mean post-course self-ratings for all three 
partner sites were 3.3 for ability to apply skills learned on the job and 3.3 for 
ability to apply skills learned in everyday life on a 5 point scale, with 5 being 
high. This indicates positive participant perceptions of progress in learning. (For 
more detail on project outcomes, please see the “Product” section of this report 
and the Appendices.) The Instructor’s Guides and participant materials 
document the instructional activities for each traditional and technology-based 
course. 

3. Are the workplace literacy program instructional decisions and activities 
generally consistent among instructors who have similar responsibilities or who 
serve the needs of similar participants? The only evidence available to the 
external evaluator for addressing this issue was that information contained in two 
of the site visit reports prepared by the previous external evaluators, Mikulecky 
and Lloyd (1996: 

• Davis Tool -‘ Instructor Priorities - All interviewees (except, of 
course, the instructor), had high praise for his commitment and 
availability to help those who were having difficulty. One participant 
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stated: ‘He’s not in a real hurry to get the heck out of here’ at the end 
of class time. The participants were also impressed by his giving out 
his home telephone number for them to call on Saturdays. The Human 
Resources Director thought that this caring attitude may have kept 
some workers attending the class. The instructor himself saw his 
priorities as evaluating student needs and attitudes, and then evaluating 
skill levels. He also mentioned the importance of feedback from and 
to students: ‘You need to let them know you care.’ Connection 
between class and the workplace - The instructor took a wider view of 
the workplace connection, dealing in class with career security and 
encouraging workers to develop skills other than for Davis through 
other educational institutions. This extended to bringing to class 
community college enrollment information. Both participants had 
picked this up and spoke of their need for more education as a 
preparation for an uncertain future. ’Davis may not survive — 
education helps’ and ‘If I need to get a new job. I’ll be better 
prepared. ’” 

• City Disposal - “ Instructor Priorities - The instructor saw her top 
priority as planning for ideas to involve the students, because she saw 
communication and confidence-building as very important. She 
commented that some students were ashamed of their inability to read 
and write, but she described their speaking and listening skills as 
‘outstanding — they surprised themselves.’ For the CDL class, she 
emphasized knowing the material (which was new to her), but pointed 
out that ‘the CDL is reading comprehension,’ which related to her 
background in reading education. She summed up her attitude to 
teaching in the words: ‘Respect yourself and respect your students.’ 
That this attitude had come across to the participants was clear from 
their comments, which included: ‘She is prepared and on time,’ and 
‘She’s a good teacher — with a problem, she’ll take time to help out.’ 
They were particularly impressed that she had taken the CDL test with 
the first class: ‘The lady cares.’ The General Manager and the Human 
Resources Director had also heard such comments and emphasized the 
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instructor’s patience, non-intimidating personality, and good rapport 
with her students. In addition, the instructor regularly prepares a 20- 
minute cassette tape to guide the work of one student who is virtually a 
non-reader. This extra effort and dedication on the instructor’s part 
has been noticed by many and was a word-of-mouth recruiting aid. 
Connection between class and the workplace - With regard to the 
Communications class, the Human Resources Director anticipated an 
increase in skills at filling out forms, route sheets, and tracking reports, 
as well as talking over the radio, noting that the class uses company 
forms. The instructor already saw changes: ‘They are eager to start the 
newsletter [a class activity] and they are more relaxed and willing to 
speak up.’” 

These reports indicate that there is a high level of commitment to the participants 
and to the program and its success across project sites. It is inferred from project 
reports that instructor training and the additional staff development sessions were 
geared toward establishing and maintaining consistent standards for instructional 
delivery across all sites and classes. Information in the Instructor Guides also 
support this inference. Monitoring of instruction and classes by the Program 
Coordinator and the Assistant Director for Instruction (Job Positions/ 
Descriptions, 1998) also support the inference that consistency across 
instructional sites was encouraged and supported. 

4. Are the participants in the workplace literacy program making the progress that 
was anticipated ? How is this determined? Participant progress was determined 
by scores on diagnostic pretests (customized course tests and the TABE Reading 
and Math tests) and parallel or identical posttests. Additional information was 
gathered by self-report of participants on the Learner Assessment and Learner 
Expectation forms used with each course. As noted in Questions 1. and 2. above, 
leaner self ratings and posttest scores all indicate general gains for each course at 
all sites. 

5. How much time is spent in instruction with workplace literacy program 
participants in whole group, small group, or independent instructional formats? 
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The only sources for addressing this issue were the two Instructor’s Guides 
reviewed by the external evaluator. Both of these detailed lesson plans 
encouraging use of all three formats for instruction. Even the technology-based 
Math for Machine Operators: GET IT IN GEAR! integrated traditional paper- 
pencil training with the computer delivered instruction, thus enabling group 
processing of information and learning. 

6. Are the workplace literacy program resources planned for use actually being 
used? Evidence from reports and databases indicates that program resources 
planned for use were used during program operations. The only exception 
appears to be the original research plan, which was modified during the project. 
The Needs Assessment Process and related instruments for data collection to 
analyze job tasks and materials for literacy skills to use in development of whole 
language (or functionally contextual) instructional materials was created, refined, 
and validated for use during the project. Standardized but comprehensive sets of 
questions were developed for use at work sites with hourly employees and 
supervisors, along with observation checklists for understanding and referencing 
shop floor observations of job tasks being performed. The strong emphasis and 
built in structure for including workers in this process demonstrated the strength 
and mutually supportive nature of the partnership between employers, labor 
organizations, and the educational providers. Procedures for gathering data and 
the development of relevant forms for impact research and for project evaluation 
provided the means for rapid, consistent, and complete responses to investigative 
inquiries, as well as formatting that facilitated entry of data into the databases 
created. Facilities, instructional materials, and division of staff responsibilities 
were developed and/or used as planned and described in the funding proposal 
(March, 1995). (Please see the “Input” section of this report for more detail on 
these project resources.) 

7. Is there a need for additional resources not initially planned for ? Specific site 
comments concerning facilities indicate that, due to the success of the program 
instructional facilities could have been expanded to accommodate more learners 
or activities. The previous external evaluators, Mikulecky and Lloyd, also noted 
in their reports that, although large amounts of impact research data had been 
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collected, analysis of it had not been completed. The current external evaluator 
found that, although the previous external evaluators had assisted the Project 
ALERT staff with developing categories for cost/benefit analysis, data were 
incomplete for comparing accomplishments with those of similar projects or for 
assigning them monetary values to perform return on investment calculations. 

For a discussion of project process, please see "Summary of Results" under the 
Discussion section of the evaluation. Following receipt of final data in August, 1998, 
PPLC assessed program outcomes (or "product") to determine the degree of project 
effectiveness. The results are addressed in the next section. 
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Project Product : 

To what extent are there indicators of project effectiveness? 

The C.I.P.P. model enables gathering of evaluation data from more than one 
source to promote triangulation of results in an attempt to arrive at valid conclusions 
concerning project effectiveness. PPLC evaluated the Project ALERT program from 
three different perspectives of users: 

• participant survey statements, coments, and self-rating statements concerning 
achievement of personal learning goals and value of the course; 

• analysis of participant pre-program and post-program assessment scores; 

• Analysis of costs versus deliverables by goal categories; 

• review of documentation related to institutionalization and/or integration of 
Project ALERT training as a part of ,or with, existing partner training courses; 
and, 

• review of documentation evidencing dissemination of project model and 
materials to other health care organizations, nation-wide. 

The following set of probes was used to guide interview questions, data collection 
and analyses: 

1. Are participants learning skill applications according to the workplace literacy 

program's definition of skill need for participant performance of critical job tasks? 

2. Do program participants continue to use skill applications after they leave class? 

3. Do program participants apply new learning to performance of critical job tasks? 

4. Do program participants view the skill applications they have learned as having 

utility? 
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5. Is each participant acquiring the skill applications for critical job tasks as 
identified in needs assessment procedures at their work sites? Is progress 
satisfactory? Has job performance on critical tasks improved? 

6. Can the partnering companies determine cost benefits derived from workplace 
literacy program instruction? 

I. Are participants learning skill applications according to the workplace literacy 
program's definition of skill need for participant performance of critical job 
tasks? The first aspect of project effectiveness was determined from data 
collected on participant pre/post-program test scores. It provided evidence of 
learning gains by participants in each course, most of which were statistically 
significant to <.01 . The Project Director furnished the external evaluator with the 
information displayed below. No individual participant or class scores were 
provided. 



Course Title 


n 


Mean Raw Scores: 


Stat Sign. 


No. who 


Testing Instruments 




Pre-Test Post -Test 


<.01 


improved 


Results for Chrysler Axle: 










Math for Machine Operators 


63 








TABE Math 


21 


7.6(51%) 9.1(61%) 


YES 


14 


Customized MMO 


48 


16.2 (54%) 20.4 (68%) 


YES 


43 


Effective Communication on the 
Computer 


110 








TABE Reading 


77 


14.9(60%) 15.9(64%) 


NO 


39 


Customized ECC 


89 


20.0 (48%) 25.0 (60%) 


YES 


68 


Custom Computer 


75 


12.6 (58%) 16.3 (77%) 


YES 


59 


Interpersonal Communication 


124 








TABE Reading 


83 


16.5 (66%) 18.0 (72%) 


YES 


28 


Customized IPC 


103 


10.9 (36%) 18.5 (62%) 


YES 


44 


Results for Davis Tool: 










Apprentice Prep 


58 








TABE Math 


34 


8.4(56%) 10.1(67%) 


YES 


22 


TABE Reading 


34 


18.4(74%) 19.7(79%) 


NO 


22 


Customized A PP 


34 


22.5 (70%) 25.7 (80%) 


YES 


26 
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Course Title 


n 


Mean Raw Scores: 


Stat Sign. 


No. who 


Testing Instruments 




Pre-Test Post -Test 


<.01 


Improved 


Results for Davis Tool (cont) 










Interpersonal Communication/ 
Problem Solving 










TABE Reading 


54 


18.6 (74%) 19.5 (78%) 


NO 


28 


Customized IPC 


54 


10.8 (36%) 18.8 (62%) 


YES 


47 


SPC Prep 










Customized SPC 1 


12 


15.0(41%) 20.4(55%) 


YES 


9 


Customized SPC2 


14 


3.6 (36%) 6.5 (65%) 


YES 


12 


TABE Math 


13 


7.7(51%) 8.7(58%) 


NO 


6 


Results for City Mgmt Corp: 










Commercial Driver* s License 










Prep 


-- 








TABE Math 


15 


2.7 (18%0 3.3 (22%) 


NO 


8 


TABE Reading 


15 


16.4 (66%0 17.1(68%) 


NO 


5 


Customized CDL 


18 


38.5(77%) 45.7(91%) 


YES 


16 


Technology for the Workplace 


-- 








TABE Reading 


16 


14.5 (58%) 16.3 (65%) 


NO 


11 


TABE Math 


16 


6.8(66%) 7.1(68%) 


NO 


9 


Customized TW 


31 


8.1(43%) 18.1(91%) 


YES 


28 


Effective Communication 










Customized EC 


10 


11.5(46%) 12.0(48%) 


NO 


4 


TABE Reading 


9 


8.2(33%) 10.8(43%) 


YES 


8 


TABE Math 


10 


4.3 (29%) 5.5 (37%) 


NO 


7 


Interpersonal Communication 
Customized IPC 


20 


16.8(56%) 21/3 (71%) 


YES 


16 


Numbers at Work 


3* 


*fNot Analyzed] 


— 


— 
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The testing results most nearly reflecting the learning of skills identified during 
the needs assessment process were the customized mastery tests for each course. 
All of these tests results evidenced statistically significant gains in courses by 
participants, except one, the Effective Communications course at the City site. 



2. Do program participants continue to use skill applications after they leave class? 
The response to this inquiry question was determined from data collected on 6- 
month follow-up post-program self-rating surveys and interviews. It concerned 
participants’ perceptions of continued improved skill application on the job and in 
everyday life situations. The post-program survey results related to skill use 
frequency were as follows: 

“Since you took the class, have you done any of the following things? ” 

A great deal-5, Frequently-4, Some-3, Very little-2, Not at all-1 



Prompt Stem: 


Chrysler Axle 
Mean (n = 51) 


City Mgmt. Corp. 
Mean (n =) 


Davis Tool 
Mean (n = ) 


Used the information on your 
job? 


3.3 


3.3 


3.4 


Used the information in everyday 
life ? 


3.6 


3.5 


3.2 


Shared the information with 
others? 


3.5 


3.5 


3.2 


Decided on new career goals? 


2.5 


2.9 


3.6 


Decided on new education 
goals? 


2.7 


3.3 


3.7 


Taken more responsibility on 
your job? 


3.0 


2.6 


3.2 


Felt that you are more likely to 
get abetter job? 


3.0 


2.9 


3.6 


Felt more satisfied with your 
job? 


3.3 


2.4 


3.3 


Felt that you are a better 
worker? 


3.7 


3.3 


3.4 



These data indicate that the mean responses were that workers perceive 
themselves as using the skills learned in the Project ALERT courses they took 
“some” on the job and in everyday life situations. It is important to note that 
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participants also reported taking “some” more responsibility on their jobs and felt 
“some” more satisfaction with their jobs at two of the three locations. 
Additionally, the courses made participants at all three locations feel that they 
were “some[what]” better workers. (Please refer to the Appendices for additional 
detail on the 6-month follow-up survey results.) 

3 . Do program participants apply new learning to performance of critical job tasks? 
Although no post-program supervisor ratings or participant observations were 
conducted to collect impact data, the 6-month follow-up self-rating participant 
surveys document perceptions of application of learning to performance of job 
tasks, as well as to everyday life situations. Mean results are displayed below for 
participant self-ratings for frequency of “better performance” and “more confident 
performance” of those skills addressed across all courses. 

“Since you took the class, do you think you are better at the following things? ” 

Rating scale: Always-5, Usually-4, Sometimes-3, Seldom-2, Never-1 



Prompt Stem: 


Chrysler Axle 
Mean (n = 51) 


City Mgmt 
Mean (n = 


Corp. 


Davis Tool 
Mean (n = ) 




On the Job 


Everyday 

Life 


On the Job 


Everyday 

Life 


On the Job 


Everyday 

Life 


Reading 


3.4 


3.5 


3.1 


3.3 


3.5 


3.5 


Listening 


4.1 


4.1 


3.9 


3.4 


3.7 


3.7 


Doing Math 


3.1 


3.0 


2.6 


2.7 


3.6 


3.6 


Speaking in Public 


3.2 


3.1 


2.0 


2.2 


3.6 


3.3 


Speaking in Private 


3.6 


3.7 


3.1 


3.2 


3.6 


3.5 


Taking Tests 


2.9 


3.1 


2.7 


2.5 


3.8 


3.8 


Understanding words 


3.5 


3.5 


3.4 


3.4 


3.7 


3.6 


Solving Problems 


3.7 


3.7 


3.7 


3.5 


4.0 


3.8 


Following directions 


4.0 


4.2 


3.9 


3.8 


4.0 


3.9 


Understanding your 
responsibilities 


4.2 


3.9 


4.1 


3.3 


3.8 


3.7 


Expressing yo ur ideas 


3.8 


2.5 


3.5 


3.4 


3.8 


3.8 


Using a computer 


2.5 


3.7 


3.2 


3.7 


2.0 


1.9 
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“ Since you took the class, are you more confident in the following things? ” 

Always-5, Usually-4, Sometimes-3, Seldom-2, Never-1 



Prompt Stem: 


Chrysler Axle 
Mean (n = 51) 


City Mgmt. Corp. 
Mean (n =) 


Davis Tool 
Mean (n = ) 




On the Job 


Evei^day 

Life 


On the Job 


Everyday 

Life 


On the Job 


Everyday 

Life 


Reading 


3.6 


3.6 


2.5 


2.7 


3.8 


3.8 


Listening 


4.1 


4.0 


3.3 


2.9 


3.6 


3.6 


Doing Math 


3.2 


3.1 


2.0 


2.1 


3.6 


3.7 


Speaking in Public 


3.3 


3.3 


1.9 


1.9 


3.6 


3.6 


Speaking in Private 


3.7 


3.8 


2.6 


2.9 


3.1 


3.1 


Taking Tests 


2.7 


2.9 


1.6 


1.9 


4.1 


4.1 


Understanding words 


3.6 


3.3 


3.0 


2.9 


3.9 


3.8 


Solving Problems 


3.9 


3.8 


3.1 


3.2 


4.2 


4.1 


Following directions 


4.0 


4.1 


3.0 


3.2 


4.4 


4.2 


Understanding your 
responsibilities 


4.2 


3.9 


3.3 


2.7 


4.1 


3.8 


Expressing your i deas 


3.8 


2.5 


2.7 


2.8 


2.1 


2.1 


Using a computer 


2.4 


3.8 


2.4 


3.0 


4.5 


4.2 



Responses ranged from “usually” to “seldom” with most reporting “sometimes” 
as the frequency of skill use both on the job and in everyday life activities. Data 
patterns indicate that those skills workers reported using most frequently across 
all sites were “listening,” “following directions,” and “solving problems.” Those 
skills reported as being used least often were “expressing your ideas” and “taking 
tests.” All skills taught, however, were reported by participants as being used to 
some extent six months after the course. (Please see Appendices for additional 
detail.) 

4. Do program participants view the skill applications they have learned as having 
utility? In both Learner Expectation and 6-month post-program survey 
comments, participants indicated that the skill applications learned had utility to 
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them. Eighty-seven percent of respondents on the post-program portion of the 
Learner Expectations instrument (n = 89) indicated that their learning goals had 
been met or “almost” met. Comments on the courses taken from the 6-month 
post-program participant surveys included the following: 

• This class was more beneficial to me. I think every employee should 
experience this class. 

• The class was very basic and the information doesn’t seem to be 
needed in my job. 

• Need to have more time on the computer. 

• Bring on the advanced classes. 

• I think it would be a great benefit for the employee to have a second 
class in effective communication on the computer. [This] would give 
the student all the confidence they would need to operate a computer. 

• Excellent. Instructors very helpful. 

• Why send us to class when you can’t follow thru it yourself. 
Management. 

• When do we start the advanced classes? 

• I learned a lot of simple ways to solve the problems, especially in the 
workplace. Sometimes we have to make the first attempts. How do 
you approach a person and what do you say to them — that is the key 
most of the time in solving a problem. 

• Excellent tool to learn how to express, understand, and receive 
collective information from/to another. 

• Excellent instructors. Very helpful. 

• This class was a huge help in my private life as well as on my job. 

• A really good concept. Communicating is the key to problem solving. 
Need [periodic] reinforcement. 

• Classes for people who have been out of school for some time is a 
wake-up call; it isn’t to learn some new things, it’s a refreshing new 
outlook. 

• “Outstanding” once people understand the purpose, which I have. 

• Class is good, but we could have [done] less math and more computer. 
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• Very good class. I think about it a lot. I think everyone should take 
Pre-CNC Math. 

• Don’t stop, more. . .time. 

• The teacher was great. . . . 

• I enjoyed the class. It gave me the ability to overlook some technical 
fears of operating computers. I can operate one now without worrying 
about erasing everything. 

• I’ve enjoyed each class that I have taken and enjoyed knowing all of 
my instructors, as well. 

• It’s a good thing for people like me. 

Participant course completion rates offer another, somewhat indirect indicator of 
the value participants assign to what is being learned. Rates of completion were 
95% at Axle, 80% at City, 103% at Davis, and 100% at Wayne Center. Certain 
courses also attracted more participants than others. Again, this is an oblique 
measure, but does contribute to the overall picture. Courses reporting the most 
participants at the three partnering sites (with available data on 329 of the 683 
total participants) were: Interpersonal Communication at Axle with 156 of the 
363 total program participants, Pre-Mobile Technology at City with 46 of the 133 
total program participants, and DAT Prep and Interpersonal Communication at 
Davis with 65 and 64, respectively, of the 159 total program participants. 
Additionally, participants were asked to rate the courses they took (5-point scale, 
with 5 as highest) and asked, “Would you recommend this course to others?" and 
“Would you be interested in taking other courses?” Results across all sites (n = 
329) showed overall course rating as 3.6. Ninety-eight percent would recommend 
the courses they took to others and seventy-nine percent expressed interest in 
taking another course. (Please see Appendices for additional detail.) 

5. Is each participant acquiring the skill applications for critical job tasks as 
identified in needs assessment procedures at their work sites? Is progress 
satisfactory? Has job performance on critical tasks improved? Because of 
project sensitivity to privacy issues for participants, no pre- or post-program data 
were collected from supervisors to determine levels of participant performance 
improvement. Most participants indicated on the 6-month follow-up surveys and 
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in their comments (see Question 4. Above) that they were using the skills learned 
or found them useful on the job. Only a few respondents mentioned the need for 
more advanced skills than what had been addressed with instruction. The gains 
on the customized tests for the courses developed under Project ALERT 
demonstrate mastery of the skills identified during needs analysis for performance 
of critical job tasks. Gains from pre-test to post-test ranged from 2% to 48%, with 
the average gain across all courses being 18.05%. (Please see table of pre-post 
test scores by site under Question 1 in this section for additional detail.) 

6. Can the partnering companies determine cost benefits derived from workplace 
literacy program instruction ? Data exist in the results of the Needs Assessment 
process conducted at each partnering company for identifying indicators to use in 
the cost-benefit analysis procedure. However, no monetary values were assigned 
to these items for calculating actual dollars saved or generated by the Project 
ALERT training courses. The list below displays suggested measurable 
indicators based on information identified by workers and front-line supervisors at 
each of the three sites: 

Chrysler Axle : 

• Quality and quantity information communicated orally when machines 
break down or there are defective parts 

• Comprehending printed safety directions and Hofler printouts for 
machine setting corrections 

• Accuracy of information entered on tally sheets for down time records 
and production of parts 

• Clarity, completeness, and conciseness of written log book entries 

• Number of incidences in which someone must read or translate for 
another employee 

• Quantities and quality of items produced 

• Scrap rates 

• Training down time and rates of mastery for upgrades. 

City Management : 
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• Quality and quantity of information communicated orally when there 
are defective parts, machine breakdowns, or inventory needs 

• Comprehending dispatch forms and printed safety directions 

• Accuracy and completeness of information on cards about machinery 
problems 

• Number of incidences of machines crashing, being reset, or voiding 
tickets due to math errors 

• Productivity: number of loads carried and miles traveled; number of 
repairs; number of good pieces produced; quality of product 

• Clarity, completeness, conciseness of written log book entries 

• Scrap rates 

Davis Tool : 

• Accuracy and completeness of written information on machine 
downtime forms 

• Comprehension of printed packaging instructions, quality control 
charts, alerts, tally sheets 

• Rate of errors in reporting tonnage of steel shipped 

• Number of errors when charting quality data 

The information above would require agreed upon standards for measurement and 
assigned dollar values in order to be used to calculate cost benefits. 

Another option in determining return on investment is to examine the costs and 
deliverables by categories according to project goals. This procedure was 
suggested by the previous evaluators, Mikulecky and Lloyd. The Project Director 
provided the current external evaluator with a list of categories that had been 
derived from project ALERT goals, and accomplishments and costs associated 
with each of them. The following information was provided funding of 
$ 1 , 245 , 923 : 



Categories: 


Admin 


Needs 

Assessmt 


Whole 
Lang Devi 


Mult Med 
Develpmt 


Implemtatn 


Documentatn 


Research 


% of Total 


13.6% 


8.2% 


10.9% 


11.7% 


28.8% 


13.9% 


12.9% 


Total $s 


$169,620. 


$101,606. 


$136,343. 


$145,990 


$358,898. 


$172,912. 


$160,553. 
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Accomplishments were described as follows: 

Administration . 

Developed and refined overall project goals 
Established “three-year vision” for deliverables 
Developed expenditure guidelines 
Hired and assigned staff as appropriate 
Located and acquired resources for staff and site use 
Developed roles and responsibilities for each staff member 
Developed detailed project plan and timelines 
Developed/maintained relationships with partner organizations 
Held biweekly project team meetings 

Conducted two-day off site retreat with partners and consultants 
Ordered and logged supplies and materials 
Developed and presented interim reports to partner organizations 
Conducted monthly Local Joint Training Committee meetings at Axle 
Organized “All Partners Meeting” to kick-off last year of grant activities 

Needs Assessment : 

Developed model for assessment process 

Designed and validated three data collection tools: Employee, Supervisor, 
Observation 

Executed data collection at three sites 
Entered data into database 

Developed summary reports for partner leaders at each site 
Created methods for utilizing data in curriculum development 

Whole Language Development 

Designed and developed lessons for 20-40 hour courses on Apprentice 
Prep, SPC Prep, Technology for the Workplace, Effective 
Communication, and Numbers at Work 
Designed and developed traditional course on fundamental 

communications (Interpersonal Communication and Problem 
Solving) 
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Developed templates for Instructor and Participant Guides 
Developed all Instructor and Participant Guides 
Presented at numerous conferences on this approach 

Multimedia Development 

Designed overall plan 

Targeted area to show best advantage at Axle 
Helped to purchase computers and software products 
Located software intellectual property for possible integration 
Purchased multimedia software for reading/writing course. 
Effective Communication on Computer 
Developed and validated Effective Communication on Computer 
Developed and validated multimedia course. Math for Machine 
Operators 

Wrote Instructor Guides for two courses 

Developed multimedia disk , Axle Process Flow (not used) 

Partially developed course. Get It in Gear 

Implementation 

UAW-Chrysler purchased 20 computers and installed them at 2 
locations. Axle and Wayne Center 
Developed recruitment plans for all sites 

Selected and trained UAW liaisons for project participation at Axle 
Hired instructors 

Organized and delivered courses at each site 
Mentored/coached instructors at sites during/after instruction 
Arranged and organized data collection procedures 
Conducted supervisor feedback sessions 
Designed and conducted 12 staff development seminars 
Assisted with courseware purchase and installation at City and 
Wayne Center 

Documented implementation process model 
Documentation 
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Documented all project staff and external meetings at partner 
organizations 

Wrote final site reports for each site 

Prepared substantial document for Axle review; organized sessions 
to demonstrate products and services to joint union- 
management team including national headquarters 

Developed numerous presentations on Whole Language 
Development for conferences 

Gave presentation at conference and wrote book chapter on 
Implementation model 

Gave presentation on multimedia development 

Data base developed 

Data entered for learner enrollment and assessment 
Data entered for 3 needs assessment instruments 
Data entered for Axle 
Data entered for City 
Data entered for Davis 



Research 

Model created 
Task list outlined 
Data collection scheme organized 
Data variables defined 

Pre-and Post-tests developed and approved for each site 
Performed literature search for related information 
Created and refined all data collection instruments 
Data analysis initiated; completed for learning gains 

The greatest expense was for implementation of the project, which 
included instructor wages and purchase of computers and software. The 
second greatest expense was for curriculum development including both 
Whole Language and Multimedia components. The third greatest expense 
was for documentation, which included monitoring activities at local sites 
and creation of the project database. The smallest percentages of funds 
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were spent on needs assessment and research. It is important to note that 
these last two categories represent one of the major project goals and one 
of major project objectives. 



Dissemination of program, program products, and research : The following 
activities have been conducted in relation to dissemination- 

Book chapter published on implementing the project model 

ERIC document produced 

Presentations on the project at Illinois Literacy Conference (1996, 1997), 
American Educational Research Association (1996), Michigan Adult 
Curriculum Connection (1996, 1997), International Society for 
Performance and Instruction (1997), Workplace Learning Conference 
(1997, 1998), Coalition of Adult Basic Education (1997), Adult Literacy 
and Technology Conference (1997), and the AECT Conference (1998). 

Appointment >to memberships in the Manufacturing Skills Standards 
Council and to the Detroit city planning group to study issues for inner 
city adults. 

Newsletters ( Education at Work) and a brochure published 

Institutionalization : Ongoing delivery of courses has been explored at Davis 

Tool, especially the Apprentice Prep course. City Management has been bought by 
another organization and is not continuing training. Chrysler Detroit Axle has reviewed 
the results and is continuing with use of some of the software, but will not continue 
training with the educational provider or the project developed products. Wayne Center 
became a stepping stone to other small companies in the area who continue to use 
products and services developed under this grant in the ongoing Empowerment Zone 
grant that dovetailed with this one. Based upon the success of this project, funding has 
been secured by WSU from the National Science Foundation to continue curriculum 
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development for use with small companies in both the Detroit and Chicago 
Empowerment Zones. 

For a discussion of program product, or outcomes, please see "Summary of 
Results" under the Discussion section of the evaluation report, which begins on the next 
page. 
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Discussion 

Limitations of this study - There was one major factor that acted as a limitation on the 
ability of this study to draw definitive conclusions from the evaluation. This factor was 
the difficulty experienced by the evaluator in collecting and obtaining some of the 
requested data due to the transition of evaluation duties after project operations had been 
completed from the originally contracted external evaluators to the current evaluator. 
Although the director in this project exhibited an exceptionally cooperative attitude, the 
unavoidable lack of contact with other key project personnel, partners, and participating 
employees functioned as a somewhat limiting factor in this evaluation. Additionally, 
some data that the current evaluator might have used as evidence to support project 
accomplishments had not been planned for or collected. 

Summary of Results - The following statements provide summary and discussion of key 
findings from the evaluation of project context, input, process, and product. 

Context - The extent to which the goals and philosophy of the project were shared by key 
project personnel and participants was found to be as follows: 

Areas of convergence : There was a good deal of consensus about program goals 
evidenced in the continuity between needs assessment data and the resulting instructional 
products and activities. Interviews with the Project Director and Program Coordinator 
indicated that there was a high level of concurrence between project key personnel on the 
goals and objectives of the project. The courses developed and implemented reflected the 
published goals of the project. Many participants also expressed learner expectations that 
coincided with the course objectives in which they were enrolled. 

Areas of divergence : The only area of divergence was found in some of the 
learner expectation goal statements for instruction. Some participants expected to acquire 
a broader range of skills than the courses addressed. This may have been due to publicity 
that was over general or recruitment methods. 

Input : The availability to the project of resources during development and 

implementation and to what extent they were effectively used was found to be as follows: 
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Strengths and Weaknesses - The curriculum materials developed for the program 
were to be built from numerous contextual examples of skill applications that employees 
at the partnering locations must use in the performance of critical job tasks. This type of 
curriculum enables participants to practice skills in ways they will use them on the job. 
The two courses reviewed, one traditional stand-up paper/pencil delivered and one 
computer-based multimedia CD-ROM delivered, contained few examples from the 
workplace. Many of the topics, worksheets, and screens dealt with pure arithmetic and 
math. The contextual workplace examples were few, but those contained in the CD- 
ROM course were of a quality that simulated workplace application of math skills used in 
CNC machine operations. The CD-ROM did, however, contain some minor “bugs” 
which need to be cleaned from the program to improve its quality. Resources for 
program development appeared to be adequate financially for the development, 
publication, and delivery of instruction. Material development time lines and on-going 
revisions based on partner input and feedback that were incorporated across the delivery 
time lines for program activities might have impacted much more greatly on the stress 
levels of less experienced developers. Despite coordination across organizations and 
facilities and among development staff, deadlines were met and almost 700 employees 
received instruction. 

Desk-top publishing the materials did not present a problem for the developers, 
nor did production of the accompanying multimedia components. Working in 
conjunction with the partners, developers were able to produce acceptable materials to 
meet union, management and employee needs.. The use of the same tests as both a pre- 
and post-assessment instruments for many modules provided a tool for measuring what 
was learned during the program and the degree of individual mastery of concepts 
presented; .however, using identical, rather than parallel instruments eliminated 
controlling for memory variables in the short period of time between measurements 
(usually less than 10 weeks). The CD-ROM testing format was short — having only 2 
practice questions and 3 test questions, as was the case for several topics in the CD-ROM 
examined, limits the ability of the test instrument to determine mastery of content. Data 
collection instruments used for participant self-assessment also created limitations on the 
validity of the data collected. Participant responses on self-appraisal scales often are 



O 
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skewed by a lack of commonly held and understood standards against which all 
participants were asked to measure their performance. 

The formatting of the program curriculum was well designed, for both instructor 
guides and participant manuals, as well as for computer screens. Formal training sessions 
for instructors and the addition of in-service seminars underscore the high level of 
commitment to project excellence that appears to have run throughout the organizations 
partnered for the project. This commitment was also evident in the facilities and 
equipment for learning provided by each site. The amount of data that were collected for 
the inquiries selected for study also reflect the cooperative nature of the partnership. 



Process : The extent to which program operations, development, and observed instruction 
were congruent with program goals and research on instructional effectiveness follows: 

Areas of convergence and divergence - Data collection records across all sites 
appeared very complete, which is unusual for most project under this funding. 
Instructors seemed to have understood their responsibility in this area for results to be so 
high. Instructors also were reported to take an “above and beyond” the job description 
stance on delivering instruction. The learning environment they created, as well as the 
curriculum development process, is in agreement with that recommended by workplace 
literacy educators and labor organizations. In A Union Approach to Workplace 
Education (1995, Labor Education and Research Center, Eugene, Oregon), it is 
recommended that, like Project ALERT, worker-centered learning be used to build on 
what workers already know, that it address the needs of the whole person, and that 
workers and their unions are active in developing and planning programs (page 27). All 
of these items concur with project published goals and were evidenced in operational 
activities. The quality of instruction was good overall. All of the developers and the 
instructors appeared to be promoting "reciprocal learning" with the participants; and 
instructors reportedly displayed a caring attitude and willingness to assist participants 
achieve their goals. 

Project management, selection and recruitment of participants, and so on, were 
well administered and appeared to be effective, with numbers of participants appropriate 
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to total populations at sites. Participant comments about what they expected to learn and 
what they actually learned were good indications that, after the program design and 
contents were determined, program advertisement and publicity, for the most part, 
accurately reflected content. 

Regularly scheduled meetings between the partners and the educational provider, 
plus the use of liaisons and active participation in these meetings by representatives of the 
partners demonstrated a high level of commitment and ability to work as a team. Topics 
addressed in the meetings and in reports appear to have allowed all partners to be kept up 
to date on project activities and progress and to give input into the decision-making 
process as options arose and were explored. 



Product : The impact of the program was assessed with a combination of indicators, 
including comments gathered from participants, comparison of pre- and post-assessment 
scores, , and evidence of dissemination. A summary of the results follows. 

Business and industry organizations normally evaluate training on four levels. 
Because workplace literacy programs are directly related to assisting workers attain 
career goals by meeting job requirements and improving performance on job tasks, it is 
appropriate to measure program outcomes using this yardstick. 

Level I - Does the proposed program match with an identified organizational need? In 
this case, the front-end analysis conducted by WSU identified organizational needs for 
improved workplace basic skills applications at Chrysler Detroit Axle, City Management 
Corporation, and Davis Tool. Comparison of the objectives and contents of the 
developed training materials with these front-end needs analyses show that specific 
critical job tasks and job materials usage were identified and targeted. The skills 
presented for instruction, although not all contextualized to the workplace, were the skills 
identified as necessary for job performance. 

Level II - Do the participants selected for training master the content of the training 
.program? Impressive, statistically significant gains from pre-/post-assessment scores. 



Prepared by Performance Plus Learning Consultants, Inc. 

6 California Avenue, Charleston, WV 253 1 1 (304)343-6861 FAX (304) 342-4996 
August, 1998 



Project ALERT- US Dept, of Education National Workplace Literacy Demonstration Project 60 

EXTERNAL EVAL UA TOR ’S FINAL REPORT 



and post-program statements by participants, compared to pre-program goal statements, 
provide strong evidence that participants mastered the content of the Project ALERT 
course materials. Post-program survey items provided extensive evidence that participant 
goals had been worked toward or met, and that these goals coincided with the objectives 
of the training programs. Gains on the assessment instrument also demonstrated training 
mastery. That gains were consistent in each category of assessment and for each of the 
courses provides an indication of the appropriateness of item design for discrimination in 
the traditional courses. To further determine item difficulty and discrimination 
capabilities, and memory interference variables, a detailed analysis of response errors 
would need to be conducted. 

The average percentage of gain was 18% on customized course mastery tests. 
When compared to gains on standardized academic basic skills tests, this is comparable 
to a gain of two stanines, usually equated with two grade levels of improvement, within 
just forty or fewer hours of instruction. Research on the transfer of learning and on basic 
skills suggests that the reason for this might be use of the functionally contextual, or 
whole language design of instructional sessions: 1, the closer the training task matches 
with the desired performance outcome, the more likely it is that positive transfer of 
learning will occur (Gick and Holyoak); and, 2, even a small amount of previous 
experience and knowledge base for the context to which the skills will apply, enables the 
learner to activate schema and raises the difficulty level of materials that can be 
comprehended by approximately 1-1.5 reading grade levels above that which would be 
achieved on a standardized reading test. Additionally, workers were asked what they 
would like to have included in training during the needs assessment process, which 
encouraged psychological buy-in through self- selection and input — which also enhances 
motivation to learn. (A complete display of assessment data can be seen in the 
Appendices.). 

Level III - Do those participants who master training demonstrate improved job 
performance in areas identified as critical, to show positive transfer of learning? 
Comments elicited from participants indicate that their perception of their personal 
learning was that it would impact on their job performance. The ability to transfer 
learning from training to performance is most dependent on the trainee's ability to 
identify performance situations to which he or she will apply the newly acquired skills. 
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along with opportunities to use the new skills and the motivation to do so (Bramley). 
Again, the use of a functionally contextual approach facilitates recognition of situations 
in which new skills can be applied to performance. The supervisor sessions to support 
learning also were important factors in using the skills beyond the classroom. That the 
participant scores demonstrate their mastery of the applied basic skill competencies they 
need to perform competently in their jobs, and that supervisors value these skills, (needs 
assessment data) provides an indication that, with motivation and opportunity to use their 
new skills, employees will, in fact, transfer what they have learned to their job 
performance. 

Level IV - Does impact on performance lead to demonstrable cost benefits, i.e., money 
saved or generated by the positive change in employee behavior? In this case, the 
partners did not report indications of positive program impact via individual behavioral 
indicators, performance appraisals, or supervisor ratings before and after training 
participation; nor did they assign values to indicators or cross-reference these with the 
instructional objectives of the program. No data exists, therefore, for determining the 
possible cost benefits derived from employee participation in the program. During 
structured interview sessions with supervisors during the needs assessment process, 
various indicators of improved job behaviors were identified. These could be studied 
with a control group design to determine possible cost benefits from the training program, 
both immediately following training and at three month intervals thereafter, to determine 
the amount of long-term gains and their retention. 

When programs are underwritten by federal funding, it is viewed in a positive 
way to apply such monies to value-added training for an organization's workers. When 
an organization does not elect to invest in continued human resource development, (i.e., 
the program), at this level of commitment beyond the funding period, it indicates that 
such training has not become an organizational priority. In information provided to the 
evaluator by the Project Director (Final Performance Report, 1998) the decision to 
institutionalize the demonstration project training courses as developed was viewed as 
highly unlikely. However, portions of the programs had been identified for continued 
delivery. The curriculum and project model for implementation was also targeted for 
continuation with other small manufacturers in the area through diffusion into the 
regional Empowerment Zone. This is strong evidence that the program is perceived to be 



Prepared by Performance Plus Learning Consultants, Inc. 

6 California Avenue, Charleston, WV 25311 (304) 343-6861 FAX (304)342-4996 
August, 1998 




Project ALERT - US Dept, of Education National Workplace Literacy Demonstration Project 62 

EXTERNAL EVALUATOR'S FINAL REPORT 



one that adds value to the Detroit-area manufacturing community of companies and labor 
organizations. 

If WSU is to continue offering Project ALERT courses as a part of other tangent 
projects in the area, the tasks and materials will need to be updated and customized 
periodically to ensure that they remain valid contextual examples of actual workplace 
situations and requirements. The process and design pattern for curriculum development 
set out by WSU in its model of whole language and multimedia instruction provides a 
proven prototype that could be replicated for development of additional exercises. The 
training and use of the materials in the future is wholly dependent upon the decisions of 
the partnering and other local companies; but the well-developed, structured, published 
instructor guidelines, participant materials and CD-ROMs ensure the flexibility of future 
delivery options. 
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Conclusions and Recommendations 

Based on the results of this evaluation, the following conclusions and 
recommendations concerning stated grant goals are offered. 

There is strong evidence showing: 

• the design, development, and implementation of innovative workplace 
literacy programs that are tailored to the skills and cultural background 
of the participants through: 

-conducting a comprehensive pre-design analysis of jobs, 
employees, and organizations to gather data requisite to the design 
of effective literacy training; 

-collaboratively designing and developing workplace literacy 
programs that are responsive to the unique needs of the 
organizations, jobs, and employees; 

-designing resource-rich classrooms that support traditional and 
technology-based instruction; 

-recruiting participants, implementing the site-specific programs, 
and providing for reinforcement and transfer of training to the 
workplace. 

• Dissemination of the program and program products; and, 

• Promotion of program institutionalization and diffusion. 



There is a moderate amount of evidence indicating: 



• Conduction of program evaluation and program impact research; and. 
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• Dissemination of research findings. 



Recommendations : 

1. Identify individual activities or responses that directly correlate with each 
learning objective or competency for each course. Use these to create 
performance-based or written items to develop parallel versions of a testing 
instrument to measure levels of skill attainment resulting from training. 

2. Develop performance indicators, from supervisor and worker input, for each 
critical job task addressed by the training courses. Use these indicators to 
benchmark pre-training levels of performance and to measure post-training 
levels of performance of participants. Use this information to determine the 
amount of impact of training on job performance and to set performance 
standards for appraisals. If monetary values are assigned to performance 
indicators, and measures established a traditional ROI formula can be used to 
calculate cost benefits or percentages of cost benefits. Also use performance 
indicator information to modify course content to include more contextual 
examples of skill usage, and to update courses over time, as needed, and to 
determine team, department, and/or partnering organization future training 
needs. 



Concluding Statement : After working with this project for only one month to review data 
and documentation of activities, it is the opinion of this evaluator that this has been one 
of the better U S Department of Education Workplace Literacy demonstration projects 
funded to date. This conclusion is based on abundant evidence showing: 1.), the vast 
amount of high quality, customized training materials that the WSU staff produced 
throughout the demonstration period; 2 ), the ability of the Project ALERT staff and 
representatives of its partners to accomplish project tasks and solve problems through 
tenacity, innovation, unflagging high energy levels, and constant attention to detail; and. 
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3 ), the pending diffusion of the project curricula and implementation model to other area 
efforts. These ingredients indicate success. 
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Project Alert Demographics: A- 1 
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Project Alert Demographics: A- 2 
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Project Alert Demographics: A- 3 
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Project Alert Demographics: 
Age 





n 


Detroit Axle: 
Effective 
Communication 
on the Computer 






' 78 


Detroit Axle: 
Interpersonal 
Communication & 
Problem Solving 






80 


Detroit Axle: 

Math for Machine 
Operators 






21 


City Disposal: 
Commercial 
Driver's License 
Prep 






19 


City Disposal: 
Technology for 
the Workplace 






17 


City Disposal: 

Effective 

Communication 






11 


City Disposal: 
Effective 
Communication, 
Part II 






6 


City Disposal: 
Interpersonal 
Communication 
and Problem 
Solving 






14 


Davis Tool: 

Apprenticeship 

Prep 






60 


Davis Tool: SPC 
Prep 






14 


Totals 


320 






% (n =320) 





3 'repared by Performance Plus LeamingConsultants, Inc. 
I ugust, 1998 



A- 4 



73 



Project Alert Demographics: 
Age 



A- 



R 





21 


22 


23 


24 


25 


26 


27 


28 


29 


30 


31 


32 


33 


34 
































20-29 yrs. old: 






























Total n 


48 




























% ofn 


14.6% 


























































30-39 yrs. old: 






























Total n 


111 




























% ofn 


33.7% 


























































40-49 yrs. old: 






























Total n 


109 




























% of n 


33.1% 


























































50-59 yrs. old: 






























Total n 


59 




























% of fl 


17.9% 


























































60-69 yrs. old: 






























Total n 


2 




























% ofn 


0.6% 






























} repared by Performance Plus LeamingConsultants, Inc. 
lugust, 1998 



74 



Project ALERT Demographics: B-1 
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Project ALERT Demographics: 
Education Levels 
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Project ALERT Demographics 
Education Levels 





GED 


some coll 


coll dgr 


trade sch 


Detroit Axle: 
Effective 
Communication 
on the Computer 












10 


21 


5 


8 


Detroit Axle: 
Interpersonal 
Communications 
and Problem 
Solving 












9 


40 


7 


22 


Detroit Axle: 

Math for the 
Machine Operator 












2 


8 


22 


4 


City Disposal: 
Commercial 
Driver's License 
Prep 












2 


4 


3 


5 


City Disposal: 
Technology for 
the Workplace 












4 


4 


0 


1 


City Disposal: 

Effective 

Communication 












0 


1 


0 


1 


City Disposal: 
Effective 
Communication, 
Part II 












0 


1 


0 


1 


City Disposal: 
Interpersonal 
Communications 
and Problem 
Solving 












3 


2 


10 


0 


Davis Tool: 

Apprenticeship 

Prep 












10 


15 


0 


14 


Davis Tool: 
SPC Prep 












2 


3 


1 


2 












TOTALS 


42 


99 


48 


58 












% of n (n = 329) 


12.8% 


30.1% 


14.6% 


17.6% 



I 



O 

me 



're pared by Performance Plus Learning Consultants , Inc. 
\ ugust , 1998 



73 




§ 

I 

I 

ft 

I 

i 



i 

§ 

i 

I 

i 

i 



Project ALERT 

Participation and Completion Rates 



Project Course # 


Site 


Course Name | 


# enrolled 


# completed 


% completed 


20801 


Axle 


EffecCom. on Comp. 


6 


8 


133% 


20802 


Axle 


EffecCom. on Comp 


8 


6 


75% 


20803 


Axle 


EffecCom. on Comp. 


6 


4 


67% 


20804 


Axle 


EffecCom. on Comp. 


9 


5 


56% 


20805 


Axle 


EffecCom. on Comp. 


8 


7 


88% 


20806 


Axle 


EffecCom. on Comp 


8 


8 


100% 


20807 


Axle 


EffecCom. on Comp 


9 


10 


111% 


20808 


Axle 


EffecCom. on Comp 


9 


8 


89% 


20809 


Axle 


EffecCom. on Comp 


8 


8 


100% 


20810 


Axle 


EffecCom. on Comp 


7 


8 


114% 


20811 


Axle 


EffecCom. on Comp 


6 


6 


100% 


20812 


Axle 


EffecCom. on Comp 


7 


7 


100% 


20813 


Axle 


EffecCom. on Comp 


4 


5 


125% 


20814 


Axle 


EffecCom. on Comp 


9 


9 


100% 


Subtotal 






104 


99 


95% 














20901 


Axle 


IPC/PrbSolv 


13 


10 


77% 


20902 


Axle 


IPC/PrbSolv 


9 


8 


89% 


20903 


Axle 


IPC/PrbSolv 


9 


7 


78% 


20904 


Axle 


IPC/PrbSolv 


13 


10 


77% 


20905 


Axle 


IPC/PrbSolv 


10 


8 


80% 


20906 


Axle 


IPC/PrbSolv 


10 


7 


70% 


20907 


Axle 


IPC/PrbSolv 


12 


12 


100% 


20908 


Axle 


IPC/PrbSolv 


12 


11 


92% 


20909 


Axle 


IPC/PrbSolv 


12 


9 


75% 


20910 


Axle 


IPC/PrbSolv 


9 


13 


144% 


20911 


Axle 


IPC/PrbSolv 


8 


8 


100% 


20912 


Axle 


IPC/PrbSolv 


9 


6 


67% 


20913 


Axle 


IPC/PrbSolv 


iT 


12 


75% 


20914 


Axle 


IPC/PrbSolv 


14 


13 


93% 


Subtotal 






156 


134 


86% 














21001 


Axle 


Pre-CNC 


7 


6 


86% 


21002 


Axle 


Pre-CNC 


11 


7 


64% 


21003 


Axle 


Pre-CNC 


9 


8 


89% 


21004 


Axle 


Pre CNC 


10 


11 


110% 


21005 


Axle 


Pre CNC 


6 


6 


100% 


21006 


Axle 


Pre CNC 


9 


10 


111% 


21007 


Axle 


Pre CNC 


8 


4“ 


50% 


21008 


Axle 


Pre CNC 


10 


8 


80% 


21009 


Axle 


Pre CNC 


9 


7 


78% 


Subtotal 






79 


67 


85% 














20815 (8 WkS) 


Axle 


Eff.Com Computer 


8 


7 


88% 


20816 (8 WkS) 


Axle 


Eff.Com Computer 


7 


2 


29% 


20817 (8 WkS) 


Axle 


Eff. Com Computer 


9 


3 


33% 


Subtotal 






24 


12 


50% 


Total for Axle Employees 




363 


312 


86% 
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E-1 



Project ALERT 

Participation and Completion Rates 



Project Course # 


Site | Course Name 


# enrolled 


# completed 


% completed 


30401 


City 


CDL-Prep 


4 


3 


75% 


30402 (Municipal) 


City 


CDL-Prep 


5 


2 


40% 


30403 


City 


CDL-Prep 


3 


1 


33% 


30404 


City 


CDL-Prep 


5 


4 


80% 


30405 


City 


CDL-Prep 


4 


4 


100% 


30406 


City 


CDL-Prep 


4 


3 


75% 


30407 


City 


CDL-Prep 


3 


2 


67% 


30408 


City 


CDL-Prep 


9 


1 


11% 


Subtotal 






37 


20 


54% 














30501 


City 


Pre-Mobile Tech 


5 


5 


100% 


30502 


City 


Pre-Mobile Tech . 


4 


4 


100% 


30503 


City 


Pre-Mobile Tech 


5 


5 


100% 


30504 


City 


Pre-Mobile Tech 


10 


7 


70% 


30505 


City 


Pre-Mobile Tech 


8 


6 


75% 


30507 


City 


Pre-Mobile Tech 


8 


4 


50% 


30508 


City 


Pre-Mobile Tech 


6 


6 


100% 


Subtotal 






46 


37 


80% 














30601 


City 


Effective Comm 


7 


7 


100% 


30602 


City 


Effective Comm 


6 


1 


17% 


30701 


City 


Effective Comm 


6 


3 


50% 


Subtotal 






19 


11 


58% 














30901 


City 


Interpersonal Com. 


11 


10 


91% 


30902 


City 


Interpersonal Com. 


11 


8 


73% 


31401 


City 


Number at Work 


6 


3 


50% 


31402 


City 


Number at Work 


3 


1 


33% 


Subtotal 






31 


22 


71% 


Total for City Employees 




133 


90 


68% 
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Project ALERT 

Participation and Completion Rates 



Project Course # 


Site 


Course Name 


# enrolled 


# completed 


% completed 


10101 


Davis 


DAT- Prep 


23 


22 


96% 


10102 


Davis 


DAT- Prep 


24 


4 


17% 


10103 


Davis 


DAT- Prep 


18 


13 


72% 


Subtotal 






65 


39 


60% 














10201 


Davis 


SPC-Prep 1 


7 


9 


129% 


10202 


Davis 


SPC-Prep 1 


8 


8 


100% 


10301 


Davis 


SPC-Prep 2 


8 


7 


88% 


10302 


Davis 


SPC-Prep 2 


7 


7 


100% 


Subtotal 






30 


31 


103% 














10901 


Davis 


Interpersonal Com. 


13 


10 


77% 


10902 


Davis 


Interpersonal Com. 


15 


16 


107% 


10903 


Davis 


Interpersonal Com. 


11 


1 


9% 


10904 


Davis 


Interpersonal Com. 


11 


1 


9% 


10905 


Davis 


Interpersonal Com. 


14 


1 


7% 


Subtotal 






64 


29 


45% 


Total for Davis Employees 




159 


99 


62% 
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Project ALERT 

Participation and Completion Rates 



E-4 



Project Course # 


Site 


Course Name 


# enrolled 


# completed 


% completed 


40901 


WC 


IPC/PrbSolv 


13 


8 


62% 


41101 


wc 


Writing lmp(WC) 


7 


2 


29% 


41201 


WC 


Ess. Of QC 


3 


3 


100% 


41301 


wc 


Rdg.Comp (Mercy) 


17 


12 


71% 


Total for Wayne Center 




40 


25 


63% 
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Project ALERT AXLE Post-Survey Results 



F-1 



1 Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Matti/Better 


Speaking in 
public/ Better 


tC on Computer 


On the Job 


506 


3 


5 


3 


5 


pC on Computer 


On the Job 


507 


3 


3 


1 


3 


^C on Computer 


On the Job 


508 


4 


5 


3 


3 


[EC on Computer 


On the Job 


514 


4 


4 


3 


2 


pC on Computer 


On the Job 


523 


4 


4 


5 


4 


pC on Computer 


On the Job 


527 


3 


4 


2 


4 


[EC on Computer 


On the Job 


528 


3 


4 


5 


3 


[EC on Computer 


On the Job 


533 


4 


4 


4 


4 


PC on Computer 


On the Job 


534 


4 


4 


4 


3 


pC on Computer 


On the Job 


537 


3 


4 


3 


3 


[EC on Computer 


On the Job 


539 


4 


5 


2 


4 


if C on Computer 


On the Job 


540 


1 


2 


1 


1 


pC on Computer 


On the Job 


545 


4 


3 


4 


5 


pC on Computer 


On the Job 


551 


3 


5 


2 


1 


] EC on Computer 


On the Job 


552 


5 


4 


3 


5 


If C on Computer 


On the Job 


554 


3 


4 




3 


pC on Computer 


On the Job 


555 


4 


3 


4 


3 


|fcC on Computer 


On the Job 


558 


2 


5 


3 


2 


EC on Computer 


On the Job 


562 


4 


5 


5 


3 


f C on Computer 


On the Job 


564 


4 


4 


4 




pC on Computer 


On the Job 


568 


5 


5 


3 


4 


^PC and Computers 


On the Job 


511 


5 


3 


4 


5 


[IPC and Problem 


On the Job 


502 


3 


4 


3 


5 


■PC and Problem 


On the Job 


503 


5 


5 


3 


5 


■PC and Problem 


On the Job 


505 


5 


5 


3 


5 


nPC and Problem 


On the Job 


510 




4 




4 


JlPC and Problem 


On the Job 


516 


4 


5 


2 




■PC and Problem 


On the Job 


517 


4 


5 


5 


4 


■PC and Problem 


On the Job 


520 


3 


3 


2 


3 


TPC and Problem 


On the Job 


521 


4 


4 


5 


4 


IPC and Problem 


On the Job 


584 


4 


5 


3 


5 


IpC and Problem 


On the Job 


589 


5 


4 


5 


3 


■PC and Problem 


On the Job 


594 


4 


5 


4 


5 


IPC and Problem 


On the Job 


598 


3 


4 


2 


3 


jPC and Problem 


On the Job 


603 


5 


5 


5 


5 


■PC and Problem 


On the Job 


606 




4 




4 


■PC and Problem 


On the Job 


607 


2 


4 


3 




TlPC and Problem 


On the Job 


608 


3 


4 


3 


3 


|IPC and Problem 


On the Job 


611 




4 




1 


■PC and Problem 


On the Job 


612 




4 




3 


*PC and Problem 


On the Job 


616 


4 


4 


4 


4 


IPC and Problem 


On the Job 


625 


4 


4 


4 


1 


|Pre CNC Math for 


On the Job 


542 


1 


1 


1 


1 


|Pre CNC Math for 


On the Job 


557 


3 


3 


4 


3 


TPre CNC Math for 


On the Job 


575 


3 


4 


4 


3 


Pre CNC Math for 


On the Job 


576 


4 


4 


3 


3 


■Pre CNC Math for 


On the Job 


577 


5 


4 


4 


3 


|Pre CNC Math for 


On the Job 


578 


3 


4 


4 


4 


Pre CNC Math for 


On the Job 


580 


5 


5 


5 


3 


Pre CNC Math for 


On the Job 


581 


2 


2 


3 


1 


|Pre CNC Math for 


On the Job 


587 


5 


4 


3 


2 


■Average Ratings: 






3.4 


4.1 


3.1 


3.2 


5= Highest 














1= Lowest 














1 
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Project ALERT AXLE Post-Survey Results 



F-2 



1 Course Name 


Speaking in 
private/ Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express your 
ideas/Better 


[EC on Computer 


5 


4 


4 


4 


5 


5 


4 


lEC on Computer 


4 


2 


3 


3 


3 


4 


4 I 


EC on Computer 


4 


4 


4 


4 


4 


4 


5 | 


EC on Computer 


3 


3 


4 


3 


4 


5 


2 1 


[EC on Computer 


4 


4 


4 


4 


4 


5 


4 


[EC on Computer 


3 


2 


2 


2 


3 


3 


4 


EC on Computer i 


3 


3 


4 


4 


3 


4 


4 


EC on Computer 


4 


3 


4 


4 


4 


4 


4 


[EC on Computer 


4 


3 


4 


3 


5 


5 


5 


[EC on Computer 


4 


3 


3 


4 


4 


4 


3 


|EC on Computer 


2 


t 


4 


3 


5 


5 


3 


|EC on Computer 


t 


t 


t 


t 


1 


t 


t 


[EC on Computer 


5 


4 


4 


5 


5 


5 


4 


[EC on Computer 


2 


2 


4 


2 


4 


4 


2 


EC on Computer 


5 


3 


5 


4 


5 


5 


5 


jEC on Computer 


4 


4 


4 


4 


4 


4 


4 


EC on Computer 


4 


3 


4 


3 


4 


4 


4 


'EC on Computer 


3 


3 


4 


4 


4 


5 


5 


EC on Computer 


5 


2 


4 


3 


5 


5 


3 


EC on Computer 


4 


4 


5 


3 


4 


4 


3 


[EC on Computer 


2 


2 


3 


2 


t 


t 


3 


[IPC and Computers 


3 


3 


4 


5 


4 


4 


3 


IPC and Problem 


5 


4 


5 


5 


5 


5 


5 


ilPC and Problem 


4 


3 


4 


4 


5 


4 


5 


IlPC and Problem 


5 


3 


5 


5 


5 


5 


5 


IPC and Problem 


4 


3 


4 


4 


4 


5 


5 


IPC and Problem 


3 


2 


3 


5 


5 


5 


4 


IlPC and Problem 


5 


4 


4 


4 


4 


4 


4 


IlPC and Problem 


3 


3 


3 


3 


4 


4 


4 


IPC and Problem 


4 


3 


4 


4 


4 


5 


3 


IPC and Problem 


2 


4 


4 


3 


4 


5 


4 


IlPC and Problem 


4 


3 


4 


5 


5 


5 


5 


IlPC and Problem 


4 


5 


4 


4 


5 


5 


4 


IlPC and Problem 


3 


2 


3 


3 


3 


4 


3 


j IPC and Problem 


5 


5 


5 


5 


5 


5 


5 


MPC and Problem 


4 


4 




4 


4 


4 


4 


IPC and Problem 


4 




3 


3 


3 


2 


3 


IPC and Problem 


3 


3 


3 


4 


5 


5 


4 


IPC and Problem 


4 






4 




3 




I IPC and Problem 


4 






4 


4 


3 


3 


IPC and Problem 


4 


4 


4 


4 


4 


4 


4 


IPC and Problem 


1 


2 


5 


4 


5 


5 


5 


Pre CNC Math for 


1 


t 


t 


t 


t 


t 


t 


Pre CNC Math for 


3 


4 


2 


4 


3 


3 


3 


Pre CNC Math for 


4 


3 


3 


4 


5 


5 


4 


Pre CNC Math for 


4 


3 


4 


4 


4 


3 


4 


Pre CNC Math for 


4 


4 


3 


3 


4 


4 


3 ! 


Pre CNC Math for 


4 


4 


4 


4 


4 


4 


4 


Pre CNC Math for 


4 


3 


5 


5 


5 


5 


5 


Pre CNC Math for 


1 


1 


2 


2 


2 


3 


4 


Pre CNC Math for 


4 


2 


4 


3 


4 


5 


5 


(Average Ratings: 


3.6 


2.9 


3.5 


3.7 


4.0 


4.2 


3.8 


5° Highest 
















1= Lowest 
















1 ~ ~ I 
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Project ALERT AXLE Post-Survey Results 



I 



F-3 





Using a 


Reading/More 


Listening/More 


Doing 

Math/More 


Speaking in 
public/More 


Speaking in 
Private/More 


Course Name 


Computer/Better 


/ Confident 


Cdonfident 


Confident 


Confident 


Confident 


|EC on Computer 


3 


4 


5 


4 


5 


5 


|EC on Computer 


5 


3 


3 


1 


3 


4 


tC on Computer 


3 


4 


3 


4 


3 


4 


EC on Computer 


3 


3 


5 


3 


3 


2 


|eC on Computer 


4 


4 


4 


5 


4 


5 


fee on Computer 


2 


3 


4 


2 


3 


4 


EC on Computer 


3 


3 


4 


5 


3 


3 


EC on Computer 


4 


4 


4 


4 


4 


4 


|EC on Computer 




5 


5 


4 


4 


4 


|EC on Computer 


3 


4 


4 


4 


3 


4 


TEC on Computer 


1 


5 


5 


2 


4 


2 


| EC on Computer 


1 


1 


3 


1 


3 


2 


|EC on Computer 


4 


5 


5 


5 


5 


5 


C on Computer 


3 


5 


5 


3 


2 




EC on Computer 


5 


5 


4 


3 


5 


5 


EC on Computer 


4 


3 


4 




3 


3 


lEC on Computer 


3 


4 


3 


4 


3 


4 


fee on Computer 


2 


3 


5 


3 


3 


3 


EC on Computer 


3 


4 


5 


4 


2 


5 


EC on Computer 




5 


4 


5 




4 


[EC on Computer 


5 


3 


3 


4 


4 


2 


IPC and Computers 


5 


4 


3 


3 


5 


3 


IPC and Problem 


3 


4 


5 


4 


5 


5 


JPC and Problem 




4 


3 


3 


5 


5 


IlPC and Problem 




5 


5 


3 


5 


5 


IPC and Problem 


2 




4 




4 


4 


IPC and Problem 


1 


4 


5 


3 


2 


3 


tiPC and Problem 


4 


4 


4 


5 


5 


5 


IlPC and Problem 


2 


3 


3 


3 


3 


3 


IPC and Problem 


4 


4 


4 


5 


4 


4 


IPC and Problem 




4 


5 


4 


3 


5 


IlPC and Problem 


2 


5 


4 


5 


2 


4 


II PC and Problem 


2 


5 


5 


5 


5 


5 


IPC and Problem 


2 


3 


3 


2 


3 


3 


IPC and Problem 




5 


5 


5 


5 


5 


IlPC and Problem 






4 




4 


4 


IlPC and Problem 




3 


4 


3 




4 


jlPC and Problem 


2 


3 


4 


3 


3 


3 


IlPC and Problem 






5 








IlPC and Problem 






4 




3 


5 


IlPC and Problem 


4 


4 


4 


4 


4 


4 


IPC and Problem 


1 


5 


4 


4 


1 


1 


Pre CNC Math for 


1 


1 


1 


1 


1 


1 


[pre CNC Math for 


5 


3 


3 


4 


3 


3 


fPre CNC Math for 


2 


4 


5 


4 


3 


5 


[Pre CNC Math for 


3 


5 


4 


3 


3 


4 


jPre CNC Math for 


4 


5 


4 


3 


3 


4 


[Pre CNC Math for 


4 


3 


4 


4 


4 


4 


'Pre CNC Math for 


3 


5 


5 


3 


5 


5 


Pre CNC Math for 


4 


2 


2 


3 


1 


1 


[Pre CNC Math for 


2 


5 


4 


3 


1 


4 


[Average Ratings: 


2.5 


3.6 


4.1 


3.2 


3.3 


3.7 


5» Highest 














1= Lowest 
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1 



I repared by Performance Plus Learning Consultants, Inc. 



05 



Project ALERT AXLE Post-Survey Results 



F-4 



Course Name 


Taking 
tests More 
Confident 


Understanding 

Words/More 

Confident 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/More 

Confident 


Understanding 

responsIbillties/More 

Confident 


Expressing your 
ideas/More 
Confident 


■EC on Computer 


4 


4 


4 


5 


5 


5 


lEC on Computer 


2 


3 


3 


2 


4 


3 


tC on Computer 


4 


4 


4 


4 


4 


5 


EC on Computer 


3 


5 


5 


5 


5 


4 


■ EC on Computer 


4 


4 


4 


4 


5 


4 


lEC on Computer 


2 


2 


3 


3 


4 


4 


EC on Computer 


3 


4 


4 


3 


4 


4 


. EC on Computer 




4 


4 


4 


4 


4 


■EC on Computer 


3 


4 


3 


5 


5 


5 


lEC on Computer 


3 


3 


4 


4 


4 


3 


EC on Computer 


1 


4 


4 


5 


5 


3 


EC on Computer 


1 


1 


1 


1 


1 


1 


[EC on Computer 


4 


5 


5 


5 


5 


5 


lEC on Computer 


2 


4 


4 


4 


4 




EC on Computer 


3 


5 


4 


5 


5 


5 


lEC on Computer 




4 


4 


4 


4 


4 


[EC on Computer 


3 


4 


3 


4 


4 


4 


lEC on Computer 


2 


5 


4 


5 


5® 


5 


EC on Computer 


2 


4 


3 


5 


5 


4 


EC on Computer 


4 


5 


4 


4 


5 


3 


[EC on Computer 


2 


3 


1 


1 


1 


3 


IPC and Computers 


3 


4 


5 


3 


4 


4 


IPC and Problem 


4 


5 


5 


5 


5 


5 


IIPC and Problem 


3 


4 


4 


5 


4 


5 


II PC and Problem 


3 


5 


5 


5 


5 


5 


IPC and Problem 


3 


4 


4 


4 


5 


5 


IPC and Problem 


2 


3 


5 


5 


5 


4 


IIPC and Problem 


4 


4 


4 


4 


4 


4 


■IPC and Problem 


3 


3 


3 


4 


4 


4 


IPC and Problem 


3 


4 


4 




5 


3 


IPC and Problem 


4 


5 


5 


IT 


4 


3 


llPC and Problem 


2 


4 


5 


5 


5 


5 


llPC and Problem 


5 


5 


4 


5 


5 


3 


IPC and Problem 


2 


3 


3 


4 


3 


3 


IPC and Problem 


5 


5 


5 


5 


5 


5 


IjPC and Problem 


4 




4 


4 


4 


4 


llPC and Problem 




3 


3 


3 


4 


3 


IPC and Problem 


1 


2 


4 


4 


4 


4 


IPC and Problem 






5 








[iPC and Problem 






5 


3 


3 


4 


IPC and Problem 


4 


4 


4 


4 


4 


4 


IPC and Problem 


3 


4 


5 


5 


5 


5 


Pre CNC Math for 


1 


1 


1 


1 


1 


1 


[Pre CNC Math for 


4 


2 


4 


3 


3 


3 


Pre CNC Math for 


3 


4 


4 


5 


5 


5 


Pre CNC Math for 


3 


4 


4 


3 


4 


4 


[Pre CNC Math for 


4 


3 


3 


4 


4 


3 


[Pre CNC Math for 


4 


4 


4 


4 


4 


4 


Pre CNC Math for 


5 


5 


5 


5 


5 




Pre CNC Math for 


1 


2 


2 


2 


3 


4 


Pre CNC Math for 




4 


2 


3 


5 


4 


[Average Ratings: 


2.7 


3.6 


3.9 


4.0 


4.2 


3.8 


15“ Highest 














|1= Lowest 














_j 
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Project ALERT AXLE Post-Survey Results 



F-5 



I Course Name 


Using a 

Computer/More 

Confident 


Used 

information on 
job 


Used 

information 


Shared 

information 


Career goals 


Education Goals 


Taken 

responsibilities 


[EC on Computer 


3 


5 


5 


5 


4 


5 


3 


[EC on Computer 


2 


5 


5 


4 


5 


4 


5 


EC on Computer 


3 


3 


3 


4 






4 


EC on Computer 




3 


2 


3 


3 




3 


EC on Computer 


4 


3 


4 


5 


3 


3 


4 


Jec on Computer 


2 


3 


3 


3 


4 


2 


4 


EC on Computer 


3 


3 


4 


3 


3 


3 


4 


EC on Computer 


4 


5 


4 


5 


4 


3 


5 


[EC on Computer 


3 




4 


5 




3 




|EC on Computer 


3 


4 


2 


2 


2 


1 


5 


|EC on Computer 


3 


1 


2 


2 


1 


1 


1 


JEC on Computer 


3 


1 


1 


3 


1 


1 


1 


[EC on Computer 


5 


4 


4 


4 


1 


1 


3 


fEC on Computer 


2 


3 


2 


2 


2 




5 


(EC on Computer 


5 


2 


5 


5 


3 


4 


2 


[EC on Computer 


4 


3 


3 


4 


1 


2 


3 


[EC on Computer 


1 


2 


2 


3 


1 


1 


1 


EC on Computer 


1 


3 


4 


5 




2 


5 


EC on Computer 


2 


3 


4 


4 


3 


2 I 


5 


EC on Computer 


5 


2 


4 




1 ~ .. . • <•" 


1 


1 


[EC on Computer 


5 


2 


2 


4 


3 


3 


2 


[1 PC and Computers 


5 


5 


3 


4 


3 


5 


5 


pc and Problem 


3 


2 


3 


3 


3 


4 


1 


[IPC and Problem 




4 


3 


4 


5 


3 


2 


JlPC and Problem 




2 


5 


4 


3 


5 


1 


nPC and Problem 


2 


4 


5 


3 




3 


2 


JlPC and Problem 


1 


5 


5 


4 




5 


4 


I IPC and Problem 


3 


4 


5 


4 


3 


3 


3 


[l PC and Problem 


2 


5 


5 


3 


3 


3 


5 


IPC and Problem 


4 


4 


4 


4 


2 


2 


4 


IPC and Problem 


2 


4 


4 


4 


4 


5 


4 


I IPC and Problem 


2 


4 


5 


4 


5 


4 


3 


II PC and Problem 




4 


4 


3 


5 


5 


4 


IPC and Problem 


2 


3 


3 


3 


2 


3 


2 


IPC and Problem 




5 


5 


5 


5 | 


5 




JlPC and Problem 




5 


4 


4 


1 


3 


3 


IPC and Problem 




3 


3 


3 


4 


4 


1 


IPC and Problem 


1 


5 


4 


5 


3 


2 


5 


IPC and Problem 




5 


5 


5 


3 


3 


2 


JlPC and Problem 




3 


4 


4 


1 


3 


3 


IPC and Problem 


3 


5 


5 


5 


3 


3 


5 


IPC and Problem 


1 


4 


4 


3 


5 


5 


5 


[Pre CNC Math for 


1 


1 


1 


1 


1 


1 


1 


|Pre CNC Math for 


5 


2 


3 


2 


3 


3 


2 


Pre CNC Math for 


2 


3 


3 


4 




4 


3 


Pre CNC Math for 


3 


3 


3 


2 


3 


3 


3 


jpre CNC Math for 


4 


3 


4 


3 


3 


3 


3 


[ Pre CNC Math for 


4 














Pre CNC Math for 


3 


4 


4 


4 


4 




5 


Pre CNC Math for 


^4 


2 


2 


1 


1 


1 


1 


Pre CNC Math for 




3 


2 


3 


3 


2 


3 


[Average Ratings: 


2.4 


3.3 


3.6 


3.5 


2.5 


2.7 


3.0 


5= Highest 
















1= Lowest 


































I 
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Project ALERT AXLE Post-Survey Results 



1 Course Name 


Better job 


Satisfied with 
the job 


Better Worker 


Rate Course 


f . C on Computer 


5 


4 


5 




fcc on Computer 


3 


5 


5 


5 


EC on Computer 


3 


4 


5 


5 


EC on Computer 


5 


3 


5 




EC on Computer 


3 


5 


5 




pC on Computer 


2 


3 


4 


4 


EC on Computer 


3 


3 


3 


5 


EC on Computer 


4 


5 


5 




EC on Computer 


3 


3 






pC on Computer 


1 


5 


4 


5 


EC on Computer 


1 


1 


1 


4 


£ C on Computer 


1 


1 


1 


4 


pC on Computer 


1 


3 


3 


4 


PC on Computer 








4 


|EC on Computer 


1 


3 


5 


5 


4fC on Computer 


2 


3 


4 


4 


pC on Computer 


1 


2 


3 


4 


pC on Computer 


3 


4 


4 


5 


|EC on Computer 


5 


5 


5 


4 


pC on Computer 


3 


3 


3 


5 


pC on Computer 


3 


2 


4 


4 


TPC and Computers 


5 


5 


4 




IPC and Problem 




2 


5 


5 . 


■PC and Problem 


5 


5 


5 




■PC and Problem 


3 


1 


1 


5 


IPC and Problem 


3 


3 


5 


4 


IPC and Problem 


4 


4 


5 


4 


|PC and Problem 


3 


5 


5 


5 


■PC and Problem 


3 


3 


5 


5 


IPC and Problem 


3 


3 


4 


4 


IPC and Problem 


4 


5 


4 




I PC and Problem 


5 


3 


5 


5 


■PC and Problem 


5 


4 


4 


5 


TlPC and Problem 


3 


3 


2 


3 


L'PC and Problem 


5 


4 


5 


5 


■PC and Problem 


3 


4 


4 


4 


(PC and Problem 


4 


2 


3 


4 


IPC and Problem 


5 


5 


5 


4 


(IPC and Problem 


3 


2 


3 




|(PC and Problem 


2 


3 


3 


5 


*PC and Problem 


2 


3 


5 


5 


IPC and Problem 


5 


4 


3 


4 


|Pre CNC Math for 


1 


1 


1 


3 


Ipre CNC Math for 


3 


3 


4 


4 


|Pre CNC Math for 


3 


2 


3 


4 


|pre CNC Math for 


3 


3 


3 


4 


|Pre CNC Math for 


3 


4 


4 




Ipre CNC Math for 








3 


Pre CNC Math for 


5 


5 


5 


5 


Pre CNC Math for 


1 


1 


1 


3 


|Pre CNC Math for 


3 


5 


3 


4 


lAverage Ratings: 


3.0 


3.3 


3.7 


3.5 


5= Highest 










1= Lowest 










1 











I 

I 
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Project ALERT AXLE Post-Survey Results 



Course Name 


Recommend 


Position 

changed 


Interested in 
another Class 


Interviewed 
about course 




[EC on Computer 


Yes 


No 


Yes 


Yes 




|EC on Computer 


Yes 


Yes 


Yes 


Yes 




tC on Computer 


Yes 


Yes 


Yes 






EC on Computer 


Yes 


No 


No 






|EC on Computer 


Yes 


No 


Yes 






fee on Computer 


Yes 


No 


Yes 


Yes 




EC on Computer 


Yes 


Yes 


Yes 


No 




EC on Computer 


Yes 


Yes 


Yes 


Yes 




fee on Computer 


Yes 


Yes 


Yes 


No 




EC on Computer 


Yes 


Yes 


No 






EC on Computer 


Yes 


Yes 


Yes 


No 




EC on Computer 


Yes 


Yes 


Yes 


Yes 




lEC on Computer 


Yes 


No 


Yes 


Yes 




fee on Computer 


Yes 






No 




EC on Computer 


Yes 


No 


Yes 


Yes 




,EC on Computer 


Yes 


No 


Yes 






[EC on Computer 


Yes 


No 


Yes 


No 




fee on Computer 


Yes 


No 


Yes 






[EC on Computer 


Yes 


No 


Yes 






fee on Computer 


Yes 


No 


Yes 


Yes 




[EC on Computer 


Yes 


No 


Yes 






IPC and Computers 


Yes 


Yes 


Yes 






IPC and Problem 


Yes 


Yes 


Yes 


Yes 




jlPC and Problem 


Yes 


Yes 


Yes 


Yes 




II PC and Problem 


Yes 


No 


Yes 


Yes 




IPC and Problem 


Yes 


No 


Yes 


No 




IPC and Problem 


Yes 


No 


Yes 


Yes 




II PC and Problem 


Yes 


No 


Yes 


Yes 




II PC and Problem 


Yes 


No 


Yes 






IPC and Problem 


Yes 


No 


Yes 


Yes 




IPC and Problem 


Yes 


Yes 


Yes 


Yes 




II PC and Problem 


Yes 


No 


No 


No 




II PC and Problem 


Yes 


No 


Yes 


Yes 




IPC and Problem 


Yes 


No 


No 


No 




IPC and Problem 


Yes 


No 


Yes 






l|PC and Problem 


Yes 


No 


Yes 






II PC and Problem 


Yes 


No 


Yes 






IPC and Problem 


Yes 


No 


Yes 


Yes 




J IPC and Problem 


Yes 


Yes 


No 


Yes 




llPC and Problem 


Yes 


No 


Yes 


Yes 




tIPC and Problem 


Yes 


No 


Yes 


No 




IPC and Problem 


Yes 


Yes 


Yes 


Yes 




|Pre CNC Math for 


No 


No 


No 


Yes 




[pre CNC Math for 


Yes 


No 


Yes 


No 




Pre CNC Math for 


Yes 


No 


Yes 


Yes 




Pre CNC Math for 


Yes 


No 


Yes 


No 




| Pre CNC Math for 


No 


Yes 


No 


No 




IPre CNC Math for 


Yes 


No 


No 


No 




Pre CNC Math for 


Yes 


Yes 


Yes 


No 




Pre CNC Math for 


No 


Yes 


Yes 


No 




j Pre CNC Math for 


Yes 


No 


Yes 


Yes 




[Average Ratings: 


46 


17 


42 


23 


Yes respons. 


5= Highest 


3 


33 


6 


15 


No respons. 


1= Lowest 


51 


50 


50 


38 


Total respon 




94% 


34% 


84% 


61% 


% Yes resp. 



i 
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Project ALERT AXLE Post-Survey Results 



I Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


Speaking in 
public/Better 


| Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


Speaking in 
public/Better 


[EC on Computer 


In Everyday Life 


506 


5 


5 


5 


5 


IEC on Computer 


In Everyday Life 


507 


4 


3 


3 


3 


EC on Computer 


In Everyday Life 


508 


4 


3 


4 


3 


EC on Computer 


In Everyday Life 


514 


3 


5 


3 


2 


IEC on Computer 


In Everyday Life 


521 


4 


4 


5 


4 


IEC on Computer 


In Everyday Life 


523 


4 


4 


5 


4 


EC on Computer 


In Everyday Life 


527 


3 


3 


2 


3 


EC on Computer 


In Everyday Life 


528 


3 


4 


4 


4 


[EC on Computer 


In Everyday Life 


533 


5 


4 


5 


4 


pC on Computer 


In Everyday Life 


534 




5 






EC on Computer 


In Everyday Life 


537 


4 


5 


4 


3 


EC on Computer 


In Everyday Life 


539 


3 


5 


4 


1 


EC on Computer 


In Everyday Life 


540 


1 


1 


1 


1 


IEC on Computer 


In Everyday Life 


545 


5 


5 


5 


5 


EC on Computer 


In Everyday Life 


551 


3 


5 


2 


1 


EC on Computer 


In Everyday Life 


552 


5 


5 


4 


5 


EC on Computer 


In Everyday Life 


554 


4 


4 




3 


fee on Computer 


In Everyday Life 


555 


5 


4 


3 


3 


EC on Computer 


In Everyday Life 


558 


4 


5 


3 


2 


EC on Computer 


In Everyday Life 


562 


3 


5 


3 


3 


[EC on Computer 


In Everyday Life 


564 


4 


4 


5 




EC on Computer 


In Everyday Life 


568 


4 


3 


2 


2 


IPC and Computers 


In Everyday Life 


511 


4 


3 


4 


5 


IIPC and Problem 


In Everyday Life 


502 


2 


4 


1 


5 


II PC and Problem 


In Everyday Life 


503 


4 


5 


3 


5 


IPC and Problem 


In Everyday Life 


505 


5 


4 




5 


IPC and Problem 


In Everyday Life 


510 




5 




4 


jlPC and Problem 


In Everyday Life 


516 


4 


5 


4 


3 


II PC and Problem 


In Everyday Life 


517 


4 


4 


5 


5 


IPC and Problem 


In Everyday Life 


520 


3 


3 


2 


3 


IPC and Problem 


In Everyday Life 


584 


4 


4 


3 


4 


LlPC and Problem 


In Everyday Life 


589 


5 


4 


5 


3 


IIPC and Problem 


In Everyday Life 


594 


4 


5 


3 


5 


IPC and Problem 


In Everyday Life 


598 


3 


3 


2 


3 


IPC and Problem 


In Everyday Life 


603 


5 


5 


5 


5 


IIPC and Problem 


In Everyday Life 


606 




4 




3 


llPC and Problem 


In Everyday Life 


607 


4 


3 






IPC and Problem 


In Everyday Life 


608 


4 


4 


5 


5 


JlPC and Problem 


In Everyday Life 


611 




4 






IlPC and Problem 


In Everyday Life 


612 




4 




2 


IPC and Problem 


In Everyday Life 


616 


4 


4 


4 


4 


IPC and Problem 


In Everyday Life 


625 


4 


4 


1 


1 


jpre CNC Math for 


In Everyday Life 


542 


1 


1 


1 


1 


jpre CNC Math for 


In Everyday Life 


557 


3 


3 


4 


3 


Pre CNC Math for 


In Everyday Life 


575 


4 


5 


4 


3 


Pre CNC Math for 


In Everyday Life 


576 


5 


4 


4 


4 


Pre CNC Math for 


In Everyday Life 


577 


5 


5 


4 


4 


[Pre CNC Math for 


In Everyday Life 


578 


4 


4 


4 


4 


Pre CNC Math for 


In Everyday Life 


580 


5 


5 


5 


3 


Pre CNC Math for 


In Everyday Life 


581 


2 


2 


3 


1 


Pre CNC Math for 


In Everyday Life 


587 


5 


4 


3 


2 


lAverage Ratings: 






3.54 


4.1 


3.02 


3.12 


5= Highest 














1= Lowest 
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Project ALERT AXLE Post-Survey Results 



G-2 



j Course Name 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express your 
ideas/Better 


I Course Name 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express your 
ideas/Better 


|EC on Computer 


5 


5 


5 


5 


5 


5 


5 


JEC on Computer 


4 


2 


3 


4 


5 


4 


4 


EC on Computer 


4 


4 




5 


4 


5 


3 


EC on Computer 


5 


3 


3 


2 


3 


3 




EC on Computer 


4 


3 


4 


4 


4 


4 


4 


fee on Computer 


5 


4 


4 


4 


5 


4 


4 


EC on Computer 


4 


2 


3 


3 


4 


4 


2 


EC on Computer 


4 


4 


3 


3 


4 


4 


4 


fee on Computer 


4 




4 


4 


5 


4 


4 


fee on Computer 




3 




3 


5 


4 


4 


EC on Computer 


3 


3 


3 


4 


4 


4 


3 


£C on Computer 


3 


1 


5 


3 


2 


2 


1 


JpC on Computer 


1 


1 


T 


1 


2 


1 


1 


fee on Computer 


5 


4 


4 


5 


5 


5 


5 i 


EC on Computer 






4 


2 


3 


2 


3 


£C on Computer 


5 


4 


5 


4 


5 


5 


5 


[EC on Computer 


4 


4 


4 


4 


4 


4 


4 


^EC on Computer 


4 


4 


4 


1 


4 


3 


2 


|EC on Computer 


3 


3 


4 


4 


5 


5 


2 


|EC on Computer 


5 


2 


4 


3 


5 


3 


3 


[EC on Computer 


4 


5 


5 


3 


4 


4 




tC on Computer 


3 


1 


2 


2 


3 


4 


4 


IRC and Computers 


4 


4 


3 


5 


5 


4 


5 


11 PC and Problem 


5 


3 


5 


5 


5 


5 


2 


II PC and Problem 


5 


3 


5 


4 


5 


4 




IPC and Problem 


5 


5 


5 


5 


5 


5 




IPC and Problem 


4 


4 


4 


4 


4 


5 


2 


llPC and Problem 


4 


3 


3 


5 


5 


4 


1 


pPC and Problem 


5 


4 


4 


4 


4 


4 


4 


IPC and Problem 


3 


3 


3 


3 


3 


3 


1 


IPC and Problem 


4 


5 


4 


3 


5 


4 


2 


llPC and Problem 


4 


3 


4 


4 


5 


5 


2 


II PC and Problem 


4 


5 


4 


4 


3 


4 


3 


IPC and Problem 


4 


2 


3 


3 


3 


4 


2 


IPC and Problem 


5 


5 


5 


5 


5 


5 




llPC and Problem 


4 


3 




4 


4 


4 




llPC and Problem 


4 




3 


4 


4 


4 




IPC and Problem 


5 


5 


4 


4 


5 


5 


2 


|IPC and Problem 


4 






4 


4 






llPC and Problem 


2 






3 


3 


4 




IPC and Problem 


4 


4 


4 


4 


5 


5 


4 


IPC and Problem 


3 


4 


5 


5 


5 


5 


1 


jpreCNC Math for 


1 


t 


1 


1 


1 


1 


1 


IPreCNC Math for 


3 


4 


3 


4 


4 


3 


5 


Pre CNC Math for 


4 


3 


3 


3 


4 


4 


3 


Pre CNC Math for 


4 


4 


5 


5 


4 


4 


3 


IPreCNC Math for 


4 


4 


4 


4 


3 


3 


4 


|pre CNC Math for 


4 


4 


4 


4 


4 


4 


4 


Pre CNC Math for 


5 


5 


5 


5 


5 


5 


3 


Pre CNC Math for 


1 


2 


2 


1 


3 


1 


4 


IPre CNC Math for 




2 


4 


3 


4 


4 


2 


■Average Ratings: 


3.74 


3.12 


3.46 


3.7 


4.18 


| 3.9 


2.54 


5= Highest 
















1= Lowest 
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su 



Project ALERT AXLE Post-Survey Results 



G-3 



Course Name 


Using a 

Computer/Better 


Reading/More 

Confident 


Listening/More 

Cdonfident 


Doing 

Math/More 

Confident 


Speaking in 
publfc/More 
Confident 


Speaking in 
Private/More 
Confident 


1 Course Name 


Using a 

Computer/Better 


Reading/More 

Confident 


Listening/More 

Cdonfident 


Doing 

Math/More 

Confident 


Speaking in 
publlc/More 
Confident 


Speaking in 
Private/More 
Confident 


|eC on Computer 


5 


5 


5 


5 


5 


5 


|EC on Computer 


3 


3 


3 


3 


3 


4 


EC on Computer 


4 






4 


4 


4 


EC on Computer 


2 


2 


2 


3 




3 


|EC on Computer 


4 


4 


4 


5 


4 


4 


■EC on Computer 


5 


4 


4 


5 


4 


5 


EC on Computer 


3 


3 


4 


3 


3 


4 


EC on Computer 


3 


3 


4 


4 


4 


4 


iEC on Computer 


5 


5 


4 


5 


4 


4 


EC on Computer 


4 


5 


5 


4 


4 


4 


EC on Computer 


4 


4 


5 


4 


4 




EC on Computer 


1 


5 


5 


2 


4 


3 


[EC on Computer 


1 


1 


1 


1 




2 


|EC on Computer 


4 


5 


5 


5 


5 


5 


EC on Computer 




5 


4 


2 


2 


3 


,EC on Computer 


4 


5 


5 


4 


5 


5 


EC on Computer 


4 


4 


4 




3 


4 


fee on Computer 


5 


5 


4 


3 


3 


4 


EC on Computer 


4 


4 


5 


3 


3 


2 


EC on Computer 


5 


4 


5 


3 


2 


5 


EC on Computer 


4 


4 


4 


5 




4 


tC on Computer 


3 


3 


3 


2 


2 


3 


1PC and Computers 


3 


4 


3 


3 


5 


3 


»IPC and Problem 


5 


4 


4 


3 


5 


5 


II PC and Problem 


5 


4 


5 


3 


5 


5 


IPC and Problem 


4 


5 






5 


5 


IPC and Problem 


5 




5 




4 


4 


II PC and Problem 


5 


4 


5 


4 


3 


4 


ll PC and Problem 


5 


4 


5 • 


5 


5 


5 


IPC and Problem 


3 


3 


3 


2 


3 


3 


IPC and Problem 


3 


4 


4 


5 


4 


4 


1IPC and Problem 


4 


5 


4 


5 


3 


4 


flPC and Problem 


5 


4 


5 


4 


5 


5 


IPC and Problem 


3 


3 


3 


2 


3 


3 


IPC and Problem 


5 


5 


5 


5 


5 


5 


■IPC and Problem 


4 




4 




3 


4 


IPC and Problem 


4 


4 


3 






4 


IPC and Problem 


4 


3 


4 


3 


3 


3 


IPC and Problem 


4 




4 






4 


llPC and Problem . 


3 




5 




2 


5 


llPC and Problem 


4 


4 


4 


4 


4 


4 " 


IPC and Problem 


4 


4 


4 


1 


1 


3 


|Pre CNC Math for 


1 


1 


1 


1 


1 


1 


[Pre CNC Math for 


3 


3 


3 


4 


3 


3 


Pre CNC Math for 


4 


4 


5 


5 


3 


4 


Pre CNC Math for 


4 


5 


5 


4 


4 


4 


Jpre CNC Math for 


3 


5 


5 


4 


4 


4 


IPreCNC Math for 


4 


4 


4 


4 


4 


4 


Pre CNC Math for 


4 


5 


5 


3 


5 


5 


Pre CNC Math for 


2 


2 


2 


3 


1 


1 


[Pre CNC Math for 




4 


4 


3 


4 


2 


lAverage Ratings: 


3.68 


3.6 


3.96 


3.1 


3.32 


3.82 


5= Highest 














1= Lowest 















I 
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COPY AVAILABLE 



Project ALERT AXLE Post-Survey Results 



G-4 



1 Course Name 


Taking 
tests More 
Confident 


Understanding 

Words/More 

Confident 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/More 

Confident 


Understanding 
res pons i bilities/More 
Confident 


Expressing your 
ideas/More 
Confident 


Course Name 


Taking 
tests More 
Confident 


Understanding 

Words/More 

Confident 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/More 

Confident 


Understanding your 
responsibilities/More 
Confident 


Expressing your 
ideas/More 
Confident 


fC on Computer 


5 


5 


5 


5 


5 


5 


pC on Computer 


2 


3 


3 


4 


4 


1 


EC on Computer 


4 




5 


5 


5 




EC on Computer 


2 


3 


3 


2 


4 




fcC on Computer 


3 


4 


4 


4 


4 


4 


■EC on Computer 


4 


4 


4 


5 


5 


4 


EC on Computer 


2 


2 


3 


4 


4 


1 


EC on Computer 


4 


3 


3 


4 


4 


4 


tC on Computer 




4 


4 


5 


5 


4 


|EC on Computer 


3 


4 


3 


5 


4 


4 


EC on Computer 




3 


4 


4 




3 


£C on Computer 


1 


5 


3 


2 


3 


3 


fee on Computer 


1 


1 


1 


4 


3 


4 


EC on Computer 


4 


5 


5 


5 


5 


5 


EC on Computer 


1 


4 


2 


2 


2 


2 


£C on Computer 


4 


5 


4 


5 


5 


5 


fee on Computer 


4 


4 


4 


4 


4 


4 


EC on Computer 


4 


4 


1 


4 


3 


2 


EC on Computer 


3 


5 


4 


5 


5 


1 


£C on Computer 


2 


4 


3 


4 


4 


3 


fee on Computer 


5 


5 


4 


5 


4 


5 


tC on Computer 


2 


4 


3 


4 


4 


5 


IPC and Computers 


4 


3 


5 


5 


4 


5 


|PC and Problem 


2 


5 


5 


5 


5 


2 


IPC and Problem 


3 


5 


4 


5 


4 




IPC and Problem 


5 


5 


5 


5 


5 




IPC and Problem 


4 


4 


4 


4 


5 


2 


IlPC and Problem 


3 


3 


5 


5 


4 


1 


llPC and Problem 


5 


4 


4 


5 


5 


3 


IPC and Problem 


3 


3 


3 


3 


3 


2 


IPC and Problem 


3 


4 


4 


4 


5 


3 


IlPC and Problem 


3 


4 


4 


5 


5 


2 


■IPC and Problem 


5 


4 


4 


3 


4 


3 


IPC and Problem 


2 


3 


3 


3 


3 


2 


IPC and Problem 


5 


5 


5 


5 


5 




IlPC and Problem 


3 




4 


4 


4 




HPC and Problem 




3 


4 


4 


4 




IPC and Problem 


2 


4 


4 


4 


4 


2 


^PC and Problem 






4 


4 






IlPC and Problem 






4 


3 


4 




IlPC and Problem 


4 


4 


4 


5 


5 


3 


IPC and Problem 


4 


5 


5 


5 


5 


1 


IPreCNC Math for 


1 


1 


1 


1 


1 


1 


IP re CNC Math for 


4 


3 


4 


4 


3 


5 


'Pre CNC Math for 


3 


3 


4 


4 


4 


3 


Pre CNC Math for 


4 


5 


5 


4 


4 


3 


IPreCNC Math for 


4 


4 


4 


3 


3 


4 


[Pre CNC Math for 


4 


4 


4 


4 


4 


4 


Pre CNC Math for 


3 


5 


5 


5 


5 


3 


Pre CNC Math for 


2 


2 


1 


3 


1 


4 


■Pre CNC Math for 




4 


3 


2 


3 




■Average Ratings: 


2.9 


3.6 


3.8 


4.14 


3.94 


2.54 


J5® Highest 














Jl= Lowest 















I I R | ( * repared by Performance Plus Learning Consultants, 
- ■ j lugust, 1998 



Inc. 
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o 



Project ALERT AXLE Post-Survey Results 



| Course Name 


Using a 

Computer/More 

Confident 


Course Name 


Using a 

Computer/More 

Confident 


EC on Computer 


5 


IeC on Computer 


2 


EC on Computer 


4 


EC on Computer 


3 


|EC on Computer 


4 


|EC on Computer 


5 


[EC on Computer 


3 


[EC on Computer 


3 


|EC on Computer 


5 


fEC on Computer 


4 


EC on Computer 


4 


EC on Computer 


2 


[EC on Computer 


4 ' * - ’ 


|EC on Computer 


5 


EC on Computer 




,EC on Computer 


4 


|EC on Computer 


4 


tC on Computer 


5 


EC on Computer 


5 


EC on Computer 


5 


EC on Computer 


5 


EC on Computer 


4 


IPC and Computers 


3 


jlPC and Problem 


5 


[l PC and Problem 


5 


(PC and Problem 


4 


IPC and Problem 


5 


j IPC and Problem 


5 


llPC and Problem 


5 


IPC and Problem 


3 


IPC and Problem 


2 


|lPC and Problem 


4 


1 IPC and Problem 


5 


IPC and Problem 


3 


IPC and Problem 


5 


llPC and Problem 


4 


IPC and Problem 


4 


IPC and Problem 


4 


IPC and Problem 


4 


[IPC and Problem 


4 


llPC and Problem 


4 


IPC and Problem 


4 


pPre CNC Math for 


1 


jPre CNC Math for 


3 


Pre CNC Math for 


4 


Pre CNC Math for 


4 


[Pre CNC Math for 


3 


I Pre CNC Math for 


4 


Pre CNC Math for 


4 


Pre CNC Math for 


2 


[Pre CNC Math for 




1 Average Ratings: 


3.84 


5= Highest 




1= Lowest 





i 



y repared by Performance Pius Learning Consultants, Inc. 



iLiyv 



iugust, 1998 




04 



H-1 



Project ALERT 6 Month CITY Post-Survey Results 



P Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


[CDL-Prep 


On the Job 


401 


3 


4 


3 


CDL-Prep 


On the Job 


402 


4 


4 


4 


ICDL-Prep 


On the Job 


406 


3 


3 




CDL-Prep 


On the Job 


407 






2 


PMT 


On the Job 


421 


3 


4 


2 


Ipmt 


On the Job 


422 


1 


3 


2 


(PMT 


On the Job 


423 


3 


5 




Ipmt 


On the Job 


425 


4 


5 


3 


ipmt 


On the Job 


428 


5 


5 


5 


fec-i 


On the Job 


451 


5 


5 


5 


EC-1 


On the Job 


452 


3 


5 


3 


IEC-1 


On the Job 


453 


3 


2 


3 


Iec-i 


On the Job 


455 


3 


5 


2 


IEC-1 


On the Job 


456 


3 


4 


3 


[ec-i 


On the Job 


457 


4 


4 


2 


lAverage Ratings: 






3.13 


3.87 


2.60 


5= Highest 












1= Lowest 

























’ repared by Performance Plus Learning Consultants, Inc. 
' ‘VM ugust, 1998 



Project ALERT 6 Month CITY Post-Survey Results 



H-2 



P Course Name 


Speaking in 
public/Better 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


[CDL-Prep 


4 


4 


3 


4 


4 


|CDL-Prep 


3 


2 


5 


3 


3 


ICDL-Prep 






3 


3 


3 


CDL-Prep 






3 




3 


|PMT 


1 


3 


2 


3 


4 


FMT 


1 


3 


3 


2 


4 


[PMT 




3 




3 




Ipmt 


2 


5 


3 


4 


3 


IpMT 


5 


5 


5 


5 


5 


EC-1 


5 


5 


5 


5 


5 


EC-1 




3 




4 


5 


lEC-1 


1 


3 


3 


3 


3 


fec-i 


1 


3 


1 


4 


5 


[EC-1 


4 


4 


2 


4 


4 


IEC-1 


3 


4 


3 


4 


4 


Average Ratings: 


2.00 


3.13 


2.73 


3.40 


3.67 


5= Highest 












1= Lowest 


























1 repared by Performance Plus Learning Consultants, Inc. 
I ugust, 1998 






H-3 



Project ALERT 6 Month CITY Post-Survey Results 



P Course Name 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express 

your 

ideas/Better 


Using a 

Computer/Better 


Reading/More 

Confident 


CDL-Prep 


4 


4 


5 


5 


3 


ICDL-Prep 


4 


4 


4 


4 


4 


IcDL-Prep 


4 


4 


4 


2 




CDL-Prep 


4 


3 


3 


5 




PMT 


5 


5 


4 


4 


3 


(PMT 


4 


2 


2 


5 


1 


[PMT 


5 


5 


1 


3 


3 


LPMT 


4 


5 


3 


2 


4 


IpMT 


5 


5 


5 


5 


5 


EC-1 


5 


5 


5 


4 




EC-1 


1 


4 


2 


1 


3 


IEC-1 


3 


3 


3 


3 


3 


Iec-i 


4 


5 


4 


1 


3 


[ec-i 


4 


4 


4 


4 


2 


[EC-1 


3 


4 


3 




4 


[Average Ratings: 


3.93 


4.13 


3.47 


3.20 


2.53 


5= Highest 












1= Lowest 
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Project ALERT 6 Month CITY Post-Survey Results 



H-4 





• Course Name 


Listening/More 

Cdonfident 


Doing 

Math/More 

Confident 


Speaking in 
public/More 
Confident 


Speaking in 
Private/More 
Confident 


Taking 

testsMore 

Confident 


Understanding 

Words/More 

Confident 




CDL-Prep 


4 


3 


4 


4 


3 


4 


■POL-Prep 


4 


4 


3 


2 


5 : 


3 


®DDL-Prep 


1 














CDL-Prep 
















PMT 


4 


3 


2 


3 


2 


4 


B’MT 


2 


2 


1 


2 


1 


3 




[PMT 


5 






3 




3 




ipMT 


5 


3 


2 


5 


3 


4 


B>mt 


5 


5 


5 


5 


5 


5 




EC-1 
















EC-1 


5 


3 




3 




4 


EC-1 


3 


3 


1 


3 


3 


3 




EC-1 


5 


2 


4 


3 


1 


4 




EC-1 


4 


2 


4 


4 


1 


4 


fcc-i 


3 




3 


3 




4 


leverage Ratings: 


3.33 


2.00 


1.93 


2.67 


1.60 


3.00 




5= Highest 
















1= Lowest 































Ve pared by Performance Plus Learning Consultants, Inc. 
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I 



Project ALERT 6 Month CITY Post-Survey Results 



H-5 





’ Course Name 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/M 

ore 

Confident 


Understanding 

your 

responsibilities/M 
ore Confident 


Expressing 

your 

ideas/More 

Confident 


Using a 

Computer/More 

Confident 


Used 

information on 
job 




CDL-Prep 


4 


4 


4 


5 


5 


5 


HCDL-Prep 


3 


4 


4 


4 


3 


2 




(CDL-Prep 
















CDL-Prep 










5 


2 




pMT 


5 


5 


5 


5 


4 


4 


PMT 


3 


2 


1 


1 


5 


4 




PMT 


2 


5 


5 


2 




3 




fMT 


3 


4 


5 


3 


2 


3 


Hpmt 


5 


5 


5 


5 


5 


5 




'EC-1 












5 




EC-1 


5 


1 


4 


2 


1 


5 


■ec-i 


3 


3 


3 


3 


3 


2 




fec-i 


5 


4 


5 


4 


1 


3 




EC-1 


4 


4 


5 


4 


2 


4 


■ec-i 


4 


4 


4 


3 




3 




^verage Ratings: 


3.07 


3.00 


3.33 


2.73 


2.40 


3.33 




5= Highest 
















1= Lowest 
































°repared by Performance Plus Learning Consultants, Inc. 
August, 1998 



09 



I 



Project ALERT 6 Month CITY Post-Survey Results 



H-6 



1 Course Name 


Used 

information 


Shared 

information 


Career goals 


Education 

Goals 


Taken 

responsibilities 


Better job 


Satisfied 
with the job 


CDL-Prep 


5 


5 


5 


5 


3 


5 


4 


CDL-Prep 


3 


4 


3 


5 


1 


3 




ICDL-Prep 




2 






1 


2 


2 


CDL-Prep 


3 


3 


3 


4 


1 


2 


1 


PMT 


4 


3 


3 


3 


3 


4 


5 


IPMT 


4 


3 


1 


3 


1 


2 


2 


PMT 


2 


3 


2 


1 


4 




1 


PMT 


2 


2 


3 


4 


4 


4 


3 


Ipmt 


5 


5 


5 


5 


5 


5 


5 


jEC-1 | 


5 


3 


3 


3 


3 


3 


3 


IEC-1 


5 


5 


3 


3 


3 


5 


2 


|ec-i 


3 


2 


3 


3 


2 


2 


2 


Iec-i 


4 


5 


5 


5 


3 


2 


3 


[ec-i 


4 


4 


2 


3 


2 


2 


2 


(EC-1 


3 


4 


3 


3 


3 


2 




(Average Ratings: 


3.47 


3.53 


2.93 


3.33 


2.60 


2.87 


2.40 


5= Highest 
















1= Lowest 

































O °repared by Performance Plus Learning Consultants, Inc. 
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Project ALERT 6 Month CITY Post-Survey Results 



H-7 



I Course Name 


Better 

Worker 


Rate Course 


Recommend 


Position 

changed 


Interested in 
another 
Class 


Interviewed 

about 

course 




CDL-Prep 


5 


5 


Yes 


Yes 


Yes 


Yes 




IcDL-Prep 




5 


Yes 


No 


No 


No 




ICDL-Prep 


2 


4 


Yes 


No 


No 


Yes 




|CDL-Prep 


3 


4 


Yes 


No 


Yes 


Yes 




(PMT 


5 




Yes 


No 


No 


No 




Ipmt 


3 


4 


Yes 


No 


Yes 


Yes 




PMT 


4 


5 


Yes 


No 


No 


Yes 




PMT 


3 


4 


Yes 


No 


Yes 


No 




Ipmt 


5 


5 


Yes 


No 


Yes 


No 




Iec-i 


5 


4 


Yes 


No 


Yes 






EC-1 


4 


3 


Yes 


No 


Yes 


Yes 




IEC-1 


2 




Yes 


Yes 




Yes 




Iec-i 


5 


4 


Yes 


No 


Yes 


Yes 




EC-1 


4 


4 


Yes 


No 


Yes 


No 




lEC-1 




3 


Yes 


No 


No 


Yes 


Yes Resp 


[Average Ratings: 


3.33 


3.60 


15 


2 


9 


9 


[5= Highest 






0 


13 


5 


5 


No Resp 


|1= Lowest 






15 


15 


14 


14 


T otal Resp 


_ 1 






100% 


13% 


64% 


64% 


% Yes Resp 
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I 



Project ALERT 6 Month CITY Post-Survey Results 



Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


CDL-Prep 


In Everyday Life 


401 


4 


4 


4 


CDL-Prep 


In Everyday Life 


402 


4 


4 


4 


|CDL-Prep 


In Everyday Life 


406 


3 


3 




jCDL-Prep 


In Everyday Life 


407 








Ipmt 


In Everyday Life 


421 


3 


4 


2 


PMT 


In Everyday Life 


422 


1 


3 


2 


PMT 


In Everyday Life 


423 


4 


5 


3 


[PMT 


In Everyday Life 


425 


3 


3 


3 


IpMT 


In Everyday Life 


428 


5 


5 


5 


EC-1 


In Everyday Life 


451 


5 


5 


5 


EC-1 


In Everyday Life 


452 


3 




3 


|ec-i 


In Everyday Life 


453 


3 


3 


4 


EC-1 


In Everyday Life 


455 


3 


5 


2 


EC-1 


In Everyday Life 


456 


4 


4 


2 


EC-1 


In Everyday Life 


457 


4 


3 


2 


[Average Ratings: 






3.27 


3.40 


2.73 


i5= Highest 












|1= Lowest 













I 

I 

I 

I 

I 

I 

I 

I 

’ repared by Performance Plus Learning Consultants, Inc. 

I I ugust, 1998 
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Project ALERT 6 Month CITY Post-Survey Results 



1-2 



1 Course Name 


Speaking in 
public/Better 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


Course Name 


Speaking in 
public/Better 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


[CDL-Prep 


5 


5 


5 


5 


5 


ICDL-Prep 


3 


4 


4 


3 


4 


CDL-Prep 






3 


3 


3 


pDL-Prep 












IPMT 


1 


3 


2 


3 


4 


PMT 


1 


3 


3 


2 


3 


PMT 




3 


3 


3 




t>MT 


3 


5 


3 


3 


3 


PMT 


5 


5 


5 


5 


5 


EC-1 


5 


5 




5 


5 


■EC-1 




3 




4 


5 


Pec-i 


3 


3 


4 


3 


3 


EC-1 


1 


3 


1 


4 


5 


|ec-i 


4 


4 


1 


5 


4 


Iec-i 


2 


2 


3 


3 


4 


[Average Ratings: 


2.20 


3.20 


2.47 


3.40 


3.53 


|5= Highest 










1 


(l= Lowest 













O 

ERIC 

MliiTMffTlTLilJ 
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1-3 



Project ALERT 6 Month CITY Post-Survey Results 



I Course Name 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express 

your 

ideas/Better 


Using a 

Computer/Better 


Reading/More 

Confident 


Course Name 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express 

your 

ideas/Better 


Using a 

Computer/Better 


Reading/More 

Confident 


LCDL-Prep 


5 


5 


5 


5 


4 


IcDL-Prep 


4 


4 


4 


4 


4 


CDL-Prep 


4 


4 


2 


3 




CDL-Prep 






5 


3 




fPMT 


5 


5 


4 


4 


3 


fPMT 


4 


2 


5 


3 


1 


jPMT 


5 


3 


3 


3 


4 


IPMT 


3 


3 


2 


2 


3 


Ipmt 


5 


5 


5 


5 


5 


EC-1 


5 


5 


3 


5 




EC-1 


3 




5 


4 


3 


|ec-i 


2 


3 


4 


3 


3 


[ec-i 


4 


5 


1 


4 


3 


EC-1 


4 


4 


3 


4 


3 


Iec-i 


4 


2 




3 


4 


Average Ratings: 


3.80 


3.33 


3.40 


3.67 


2.67 


5= Highest 












1= Lowest 













Prepared by Performance Plus Learning Consultants, Inc. 
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Project ALERT 6 Month CITY Post-Survey Results 



1-4 







Doing 


Speaking in 


Speaking in 


Taking 


Understanding 




Listening/More 


Math/More 


public/More 


Private/More 


testsMore 


Words/More 


1 Course Name 


Cdonfident 


Confident 


Confident 


Confident 


Confident 


Confident 






Doing 


Speaking in 


Speaking in 


Taking 


Understanding 




Listening/More 


Math/More 


public/More 


Private/More 


testsMore 


Words/More 


| Course Name 


Cdonfident 


Confident 


Confident 


Confident 


Confident 


Confident 


CDL-Prep 


4 


4 


5 


5 


5 


5 


CDL-Prep 


4 


4 


3 


4 


4 


3 


ICDL-Prep 














iCDL-Prep 














IPMT 


4 


2 


1 


3 


2 


3 


PMT 


3 


2 


1 


3 


1 


2 


PMT 


5 






3 




3 


PMT 


3 


3 


3 


5 


3 


3 


fc>MT 


5 


5 


5 


5 


5 


5 


Iec-i 














|ec-i 




3 




3 




4 


EC-1 


3 


4 


3 


3 


3 


3 


EC-1 


5 


2 


1 


3 


4 


4 


IEC-1 


5 


3 


5 


4 


1 


5 


EC-1 


3 




2 


2 




3 


[Average Ratings: 


2.93 


2.13 


1.93 


2.87 


L87i 2.87 


45= Highest 






j 




1 


|l= Lowest 




1 







} repared by Performance Plus Learning Consultants, Inc. 
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I 



Project ALERT 6 Month CITY Post-Survey Results 



Course Name 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/M 

ore 

Confident 


Understanding 

your 

responsibilities/M 
ore Confident 


Expressing 

your 

ideas/More 

Confident 


Using a 

Computer/More 

Confident 


Course Name 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/M 

ore 

Confident 


Understanding 

your 

responsibilities/M 
ore Confident 


Expressing 

your 

ideas/More 

Confident 


Using a 

Computer/More 

Confident 


CDL-Prep 


5 


5 


5 


5 


5 


CDL-Prep 


4 


4 


4 


4 


3 


CDL-Prep 












ICDL-Prep 








5 


3 


IPMT 


4 


5 


5 


4 


4 


IPMT 


3 


4 


2 


5 


3 


[PMT 


2 


5 


3 




3 


IPMT 


3 


3 


3 


2 


2 


PMT 


5 


5 


5 


5 


5 


EC-1 












EC-1 


5 


3 




5 


4 


EC-1 


O 

o 


2 


2 


4 


2 


EC-1 


5 


4 


5 


1 


4 


EC-1 


5 


4 


5 


3 


4 


EC-1 


4 


4 


2 




3 


Average Ratings: 


3,20 


i 3.20 i 2,73 


2.87 


3.00 


5= Highest j 


i 




i 


1= Lowest | 


i 


1 

i 
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lugust, 1998 



10G 



I 



Project ALERT 6 Month DAVIS Post-Survey Results 



J-1 



1 Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


l Apprentice Prep 


On the Job 


226 


4 


4 


2 


f Apprentice Prep 


On the Job 


227 


2 


3 


5 


Apprentice Prep 


On the Job 


228 


3 


2 


5 


Apprentice Prep 


On the Job 


109 


4 


4 


2 


! Apprentice Prep 


On the Job 


111 


2 


2 


2 


1 Apprentice Prep 


On the Job 


112 


4 


5 


3 


Apprentice Prep 


On the Job 


122 


3 


3 


1 


I Apprentice Prep 


On the Job 


201 


5 


4 


5 


1 Apprentice Prep 


On the Job 


203 


4 


4 


4 


j Apprentice Prep 


On the Job 


212 


5 


5 


5 


I Apprentice Prep 


On the Job 


219 


5 


5 


5 


I Apprentice Prep 


On the Job 


220 


3 


2 


3 


Apprentice Prep 


On the Job 


221 


5 


5 


4 


Apprentice Prep 


On the Job 


222 




3 


5 


1 Apprentice Prep 


On the Job 


225 


4 


4 


5 


Apprentice Prep 


On the Job 


303 


3 


5 


1 


Apprentice Prep 


On the Job 


307 


3 


3 


5 


1 Apprentice Prep 


On the Job 


307 


4 


4 


4 


1 Apprentice Prep 


On the Job 


314 


3 


4 


2 


j Average Ratings: 






3.47 


3.74 


3.58 


| 5= Highest 












I 1= Lowest 

























I 

I 

I 

I 

I 

BEST COPY AVAtlftBL 1 

I ini 

D repared by Performance Plus LeamingConsultants, Inc. ^ i 

|ERJC 4 ^ 1998 



J-2 



I 



Project ALERT 6 Month DAVIS Post-Survey Results 





Speaking in 
public/Better 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


Apprentice Prep 


3 


4 


2 


3 


3 


Apprentice Prep 


4 


1 


5 


4 


5 


Apprentice Prep 


5 


5 


5 


4 


3 


Apprentice Prep 


2 


3 


4 


4 


4 


Apprentice Prep 


2 


2 


2 


2 


2 


Apprentice Prep 


5 


5 


5 


5 


5 


1 Apprentice Prep 


3 


4 


1 


3 


3 


i Apprentice Prep 


4 


5 


4 


4 


4 


Apprentice Prep 


4 


4 


4 


4 


4 


Apprentice Prep 


4 


4 


5 


5 


5 


i Apprentice Prep 


3 


5 


5 


5 


5 


’ Apprentice Prep 


3 


2 


3 


3 


3 


Apprentice Prep 


5 


5 


5 


5 


5 


I Apprentice Prep 


2 


3 


4 




5 


! Apprentice Prep 


5 


4 


5 


4 


4 


Apprentice Prep 


5 


1 


4 


5 


4 


Apprentice Prep 


3 


3 


4 


4 


4 


| Apprentice Prep 


4 


4 


5 


4 


4 


Apprentice Prep 


3 


4 




2 


4 


1 Average Ratings: 


3.63 


3.58 


3.79 


3.68 


4.00 


5= Highest 












1 1= Lowest 

























103 



ERJC 
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Project ALERT 6 Month DAVIS Post-Survey Results 



J-3 



Course Name 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express your 
ideas/Better 


Using a 

Computer/Better 


Reading/More 

Confident 


1 Apprentice Prep 


4 


5 


5 


3 


3 


r Apprentice Prep 


3 


1 


2 


1 


5 


Apprentice Prep 


4 


4 


4 


2 


4 


I Apprentice Prep 


4 


4 


3 




4 


1 Apprentice Prep 


2 


2 


2 


2 


2 


Apprentice Prep 


4 


5 


4 


5 


5 


Apprentice Prep 


3 


3 


3 


1 


3 


I Apprentice Prep 


4 


5 


4 


3 


4 


1 Apprentice Prep 


4 


4 


4 


4 


3 


| Apprentice Prep 


5 


5 


4 


2 


5 


I Apprentice Prep 


5 


5 


5 


3 


5 


1 Apprentice Prep 


4 


3 


3 


1 


3 


Apprentice Prep 


5 


5 


5 


4 


5 


1 Apprentice Prep 


4 




4 






1 Apprentice Prep 


4 


4 


5 


4 


5 


[ Apprentice Prep 


5 


5 


5 


1 


5 


[ Apprentice Prep 


4 


3 


3 


2 


3 


1 Apprentice Prep 


4 


5 


4 




4 


Apprentice Prep 


4 


4 


3 




4 


Average Ratings: 


4.00 


3.79 


3.79 


2.00 


3.79 


| 5= Highest 












| 1= Lowest 


























I 

I 

I 

I 

I 

I 

I 

O 'repared by Performance Plus LeamingConsultants, Inc. 
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J-4 



I 



Project ALERT 6 Month DAVIS Post-Survey Results 



Course Name 


Listening/More 

Cdonfident 


Doing 

Math/More 

Confident 


Speaking in 
public/More 
Confident 


Speaking in 
Private/More 
Confident 


Taking 

testsMore 

Confident 


| Apprentice Prep 


2 


3 


4 


3 


2 


1 Apprentice Prep 


3 


5 


3 


1 


5 


Apprentice Prep 


3 


4 


3 


3 


5 


L Apprentice Prep 


4 


4 


3 


2 


4 


I Apprentice Prep 


2 


2 


2 


2 


2 


Apprentice Prep 


4 


3 


5 


4 


5 


. Apprentice Prep 


3 


3 


3 


4 


3 


■ Apprentice Prep 


5 


5 


5 


5 


3 


1 Apprentice Prep 


3 


2 


2 


2 


3 


Apprentice Prep 


5 


5 


5 


2 


5 


1 Apprentice Prep 


5 


5 


4 


5 


5 


1 Apprentice Prep 


3 


4 


3 


3 


3 


1 Apprentice Prep 


4 


5 


5 


5 


5 


I Apprentice Prep 


3 


5 


2 




5 


1 Apprentice Prep 


4 




5 


4 


5 


Apprentice Prep 


5 


1 


5 


4 


5 


Apprentice Prep 


3 


5 


3 


3 


4 


Apprentice Prep 


4 


4 


4 


4 


5 


Apprentice Prep 


4 


3 


3 


3 


3 


, Average Ratings: 


3.63 


3.58 


3.63 


3.11 


4.05 


5= Highest 












1= Lowest 
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Project ALERT 6 Month DAVIS Post-Survey Results 



J-5 



Course Name 


Understanding 

Words/More 

Confident 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/More 

Confident 


Understanding 

your 

responsibilities/ 
More Confident 


Expressing 
your ideas/More 
Confident 


Apprentice Prep 


4 


4 


4 


4 


4 


Apprentice Prep 


5 


4 


5 


3 


1 


Apprentice Prep 


4 


4 


4 


3 


1 


Apprentice Prep 


4 


5 


5 


4 




Apprentice Prep 


2 


2 


2 


2 


2 


Apprentice Prep 


5 


5 


5 


5 


5 


Apprentice Prep 


3 


3 


4 


4 


1 


I Apprentice Prep 


4 


4 


5 


4 


3 


1 Apprentice Prep 


3 


4 


4 


4 


4 


1 Apprentice Prep 


5 


5 


5 


5 


2 


I Apprentice Prep 


5 


5 


5 


5 


3 


1 Apprentice Prep 


4 


3 


3 


3 


1 


1 Apprentice Prep 


5 


5 


5 


5 


5 


Apprentice Prep 




5 


5 


5 




Apprentice Prep 


5 


5 


5 


5 


4 


Apprentice Prep 


5 


4 


4 


5 


1 


Apprentice Prep 


4 


4 


5 


3 


2 


Apprentice Prep 


4 


4 


4 


4 




Apprentice Prep 


3 


4 


4 


4 




Average Ratings: 


3.89 


4.16 


4.37 


4.05 


2.05 


5= Highest 












| 1= Lowest 












1 ___ 













I 

I 

I 

I 

I 

I 

I 

| ill 

O Prepared by Performance Plus LearrtingConsultants , Inc. 

I I RJO gttrt. 1998 



Project ALERT 6 Month DAVIS Post-Survey Results 



J-6 



I 



Course Name 


Using a 

Computer/More 

Confident 


Used 

information on 
job 


Used 

information 


Shared 

information 


Career goals 


| Apprentice Prep 


4 


5 


5 


5 


4 


I Apprentice Prep 


5 


5 


5 


5 


5 


Apprentice Prep 


4 


5 


3 


3 


3 


. Apprentice Prep 


5 


2 


3 


3 


4 


I Apprentice Prep 


2 


3 


3 


3 


3 


* Apprentice Prep 


5 


4 


4 


4 


5 


Apprentice Prep 


4 


3 


3 


4 


4 


1 Apprentice Prep 


5 










I Apprentice Prep 


4 


2 


2 


3 


3 


Apprentice Prep 


5 


4 


1 


1 


1 


1 Apprentice Prep 


5 


5 


4 


3 


5 


1 Apprentice Prep 


3 


3 


3 


3 


4 


1 Apprentice Prep 


5 


1 


3 


3 


4 


| Apprentice Prep 


5 


3 


5 


4 


4 


E Apprentice Prep 


4 


5 


5 


4 


5 


Apprentice Prep 


4 


3 


2 


3 


3 


. Apprentice Prep 


4 


4 


4 


4 


4 


1 Apprentice Prep 


4 


4 


4 


4 


5 


Apprentice Prep 


4 


3 


2 


2 


3 


Average Ratings: 


4.26 


3.37 


3.21 


3.21 


3.63 


1 5= Highest 












1 1= Lowest 












1 













I 

I 

I 

I 

I 

I 

I 



I 



O n repared by Performance Plus LeamingConsultants ; Inc. 
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I 



Project ALERT 6 Month DAVIS Post-Survey Results 



J-7 





Education 

Goals 


Taken 

responsibilities 


Better job 


Satisfied with 
the job 


Better Worker 


| Apprentice Prep 


5 


3 


3 


3 


3 


1 Apprentice Prep 


5 


1 


5 


5 


3 


Apprentice Prep 


3 


2 


3 


5 


5 


Apprentice Prep 


4 


2 


5 


2 


3 


1 Apprentice Prep 


3 


3 


3 


3 


3 


Apprentice Prep 


5 


4 


4 


4 


4 


■ Apprentice Prep 


5 


5 


5 


4 


3 


Apprentice Prep 












Apprentice Prep 


3 


4 


4 


4 


4 


Apprentice Prep 


1 


5 


5 


1 


1 


Apprentice Prep 


5 


5 


5 


4 


5 


^ Apprentice Prep 


3 


2 


3 


3 


3 


Apprentice Prep 


5 


2 


5 


5 


5 


! Apprentice Prep 




3 




3 


3 


! Apprentice Prep 


5 


5 


5 


5 


5 


Apprentice Prep 


4 


2 


3 


3 


3 


Apprentice Prep 


5 


2 


2 


2 


4 


Apprentice Prep 


5 


5 


5 


5 


5 


Apprentice Prep 


5 


5 


3 


2 


2 


Average Ratings: 


3.74 


3.16 


3.58 


3.32 


3.37 


5= Highest 












1= Lowest 

























1 i q 
1 x o 



'repared by Performance Plus LeamingConsultants, Inc. 
1998 
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I 



Project ALERT 6 Month DAVIS Post-Survey Results 



I Course Name 


Rate DAT-Prep 


Recommend 


Position 

changed 


Interested in 
another Class 


Interviewed 
about DAT-Prep 


1 Apprentice Prep 


3 


Yes 


No 


Yes 


Yes 


1 Apprentice Prep 


5 


Yes 


No 


Yes 


Yes 


Apprentice Prep 


5 


Yes 


No 


Yes 


Yes 


I Apprentice Prep 


5 


Yes 


No 


Yes 


No 


I Apprentice Prep 


1 










[ Apprentice Prep 


5 


Yes 


No 


Yes 


Yes 


L Apprentice Prep 


4 


Yes 


No 


Yes 


Yes 


E Apprentice Prep 












Apprentice Prep 


4 


Yes 


Yes 


Yes 


Yes 


Apprentice Prep 


5 


Yes 


No 


Yes 


Yes 


1 Apprentice Prep 


4 


Yes 


Yes 


No 


Yes 


P Apprentice Prep 


4 


Yes 


No 


No 


No 


1 Apprentice Prep 


5 


Yes 


No 


Yes 


No 


I Apprentice Prep 


5 


Yes 


No 


Yes 


No 


1 Apprentice Prep 


5 


Yes 


Yes 


Yes 


Yes 


Apprentice Prep 


3 


Yes 


No 


Yes 


Yes 


1 Apprentice Prep 


4 


Yes 


No 


Yes 


No 


1 Apprentice Prep 


5 


Yes 


Yes 


Yes 




j Apprentice Prep 




Yes 


No 


Yes 


Yes 


L Average Ratings: 


3.79 


17 


4 


15 


11 


1 5= Highest 




0 


13 


2 


5 


1= Lowest 




17 


17 


17 


16 






100% 


24% 


88% 


69% 



114 
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Project ALERT 6 Month DAVIS Post-Survey Results 



J-9 



I 



Course Name 


Do you have 
any questions 
about the DAT 
Prep Calss 




! Apprentice Prep 


No 




f Apprentice Prep 


No 




Apprentice Prep 


NO 




r Apprentice Prep 


NO 




| Apprentice Prep 


NO 




Apprentice Prep 


NO 




L Apprentice Prep 


No 




I Apprentice Prep 


No 




Apprentice Prep 


No 




Apprentice Prep 


No 




[ Apprentice Prep 


NO 




| Apprentice Prep 


No 




Apprentice Prep 


NO 




| Apprentice Prep 


NO 




I Apprentice Prep 


No 




Apprentice Prep 


NO 




t Apprentice Prep 


NO 




1 Apprentice Prep 


NO 




[ Apprentice Prep 


NO 




l Average Ratings: 


0 


Yes Resp. 


5= Highest 


19 


No Resp. 


* 1= Lowest 


19 


Total Resp. 






% Yes Resp. 



'repared by Performance Plus LeamingConsultants, Inc. 1 j tX 
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I 



Project ALERT 6 Month DAVIS Post-Survey Results 



K-1 



1 

' 

1 


Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


t Course Name 


Field 


Participant ID 


Reading/Better 


Listening/Better 


Doing 

Math/Better 


1 Apprentice Prep 


In Everyday Life 


226 


4 


4 


2 


Apprentice Prep 


In Everyday Life 


227 


2 


3 


5 


I Apprentice Prep 


In Everyday Life 


228 


4 


5 


5 


[ Apprentice Prep 


In Everyday Life 


109 


3 


4 


4 


Apprentice Prep 


In Everyday Life 


111 


2 


2 


2 


1 Apprentice Prep 


In Everyday Life 


112 


5 


4 


3 


Apprentice Prep 


In Everyday Life 


122 


3 


3 


1 


Apprentice Prep 


In Everyday Life 


201 


4 


4 


3 


. Apprentice Prep 


In Everyday Life 


203 


3 


3 


3 


Apprentice Prep 


In Everyday Life 


212 


4 


5 


5 


Apprentice Prep 


In Everyday Life 


219 


5 


5 


5 


Apprentice Prep 


In Everyday Life 


220 


3 


2 


3 


Apprentice Prep 


In Everyday Life 


221 


5 


5 


5 


1 Apprentice Prep 


In Everyday Life 


222 




3 


5 


Apprentice Prep 


In Everyday Life 


225 


3 


4 


5 


| Apprentice Prep 


In Everyday Life 


303 


1 


3 


1 


| Apprentice Prep 


In Everyday Life 


307 


4 


3 


5 


Apprentice Prep 


In Everyday Life 


307 


4 


4 


4 


| Apprentice Prep 


In Everyday Life 


314 


3 


4 


3 




[Average Rating: 






3.26 


3.68 


3.63 




5= Highest 












1= Lowest 
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Project ALERT 6 Month DAVTS Post-Survey Results 



K 



Course Name 


Speaking in 
public/Better 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


| Course Name 


Speaking in 
public/Better 


Speaking in 
private/Better 


Taking 

tests/Better 


Understanding 

Words/Better 


Solving 

Problems/Better 


Apprentice Prep 


3 


4 


2 


3 


2 


■ Apprentice Prep 


4 


1 


5 


4 


5 


Apprentice Prep 


5 


5 


5 


4 


3 


Apprentice Prep 


3 


3 


5 


4 


4 


Apprentice Prep 


2 


2 


2 


2 


2 


I Apprentice Prep 


5 


4 


5 


5 


5 


f Apprentice Prep 


1 


5 


1 


2 


3 


j Apprentice Prep 


4 


5 


3 


4 


4 


I Apprentice Prep 


3 


3 


3 


3 


3 


1 Apprentice Prep 


4 


4 


5 


5 


5 


Apprentice Prep 


3 


5 


5 


5 


4 


| Apprentice Prep 


2 


2 


3 


3 


3 


j Apprentice Prep 


5 


5 


3 


5 


5 


Apprentice Prep 


2 


3 


4 




5 


Apprentice Prep 


4 


4 


5 


4 


4 


I Apprentice Prep 


4 


1 


4 


5 


4 


r Apprentice Prep 


2 


2 


4 


4 


3 


I Apprentice Prep 


4 


4 


5 


4 


4 


| Apprentice Prep 


3 


4 


3 


2 


4 


[Average Rating: 


3.32 


3.47 


3.79 


3.58 


3.79 


5= Highest 












|l= Lowest 












1 _ 













i 
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I 



Project ALERT 6 Month DAVIS Post-Survey Results 



K-3 



1 Course Name 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express your 
ideas/Better 


Using a 

Computer/Better 


Reading/More 

Confident 


I Course Name 


Following 

directions/Better 


Understanding 

responsibilities/ 

Better 


Express your 
ideas/Better 


Using a 

Computer/Better 


Reading/More 

Confident 


| Apprentice Prep 


4 


5 


5 


3 


3 


L Apprentice Prep 


3 


1 


3 


1 


5 


| Apprentice Prep 


4 


4 


4 


2 


4 


Apprentice Prep 


4 


4 


3 




4 


Apprentice Prep 


2 


2 


2 


2 


2 


1 Apprentice Prep 


5 


5 


5 


5 


5 


P Apprentice Prep 


3 


4 


4 


1 


3 


j Apprentice Prep 


3 


4 


4 


3 


4 


1 Apprentice Prep 


4 


4 


4 


3 


3 


I Apprentice Prep 


5 


5 


4 


2 


5 


Apprentice Prep 


5 


5 


5 


3 


5 


1 Apprentice Prep 


4 


3 


3 


1 


4 


1 Apprentice Prep 


5 


5 


5 


4 


5 


[ Apprentice Prep 


4 




4 






L Apprentice Prep 


4 


4 


4 


3 


4 


1 Apprentice Prep 


4 


4 


4 


1 


5 


Apprentice Prep 


3 


3 


3 


2 


4 


Apprentice Prep 


4 


5 


4 




4 


I Apprentice Prep 


4" 


4 


3 




4 


[Average Rating: 


3.89 


3.74 


3.84 


1.89 


3.84 


1®= Highest 












|1= Lowest 
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I 



Project ALERT 6 Month DAVIS Post-Survey Results 



K-4 



1 


Course Name 


Listening/More 

Cdonfident 


Doing 

Math/More 

Confident 


Speaking in 
public/More 
Confident 


Speaking in 
Private/More 
Confident 


Taking 

testsMore 

Confident 


Course Name 


Listening/More 

Cdonfident 


Doing 

Math/More 

Confident 


Speaking in 
public/More 
Confident 


Speaking in 
Private/More 
Confident 


Taking 

testsMore 

Confident 


Apprentice Prep 


2 


3 


4 


3 


2 


Apprentice Prep 


3 


5 


3 


1 


5 


Apprentice Prep 


3 


4 


3 


3 


5 


Apprentice Prep 


4 


5 


4 


2 


4 


Apprentice Prep 


2 


2 


2 


2 


2 


Apprentice Prep 


4 


3 


5 


5 


5 


Apprentice Prep 


3 


3 


3 


4 


3 


Apprentice Prep 


4 


4 


5 


5 


3 


Apprentice Prep 


3 


1 


3 


1 


3 


Apprentice Prep 


5 


5 


5 


2 


5 


Apprentice Prep 


5 


5 


3 


5 


5 


Apprentice Prep 


3 


4 


3 


3 


3 


Apprentice Prep 


5 


5 


5 


5 


5 


Apprentice Prep 


3 


5 


2 




5 


! Apprentice Prep 


4 


5 


4 


4 


5 


Apprentice Prep 


4 


1 


4 


3 


5 


Apprentice Prep 


3 


4 


3 


3 


4 


Apprentice Prep 


4 


4 


4 


4 


5 


Apprentice Prep 


4 


3 


3 


3 


3 


Average Rating: 


3.58 


3.74 


3.58 


3.05 


4.05 


5= Highest 












1= Lowest 

























O 

ERLC 

•i'I’LT.MiTLJ 
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Project ALERT 6 Month DAVIS Post-Survey Results 



Course Name 


Understanding 

Words/More 

Confident 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/More 

Confident 


Understanding 

your 

responsibilities/ 
More Confident 


Expressing 
your ideas/More 
Confident 


[ Course Name 


Understanding 

Words/More 

Confident 


Solving 

Problems/More 

Cdonfident 


Following 

Directions/More 

Confident 


Understanding 

your 

responsibilities/ 
More Confident 


Expressing 
your ideas/More 
Confident 


Apprentice Prep 


3 


4 


4 


4 


4 


■ Apprentice Prep 


5 


4 


5 


3 


1 


1 Apprentice Prep 


4 


4 


4 


3 


1 


Apprentice Prep 


4 


5 


5 
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Apprentice Prep 


2 


2 


2 


2 
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I Apprentice Prep 


5 


4 


5 


5 
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p Apprentice Prep 


3 


3 


4 
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1 
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4 
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4 
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3 


3 
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3 
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5 
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5 


5 
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1 Apprentice Prep 


5 


4 


3 


l 5 


1 


Apprentice Prep 


4 


4 


4 


3 


2 


, Apprentice Prepl 


4 


4 


4 


4 




Apprentice Prep 


3 


4 


4 


4 




Average Rating: 


3.84 


4.05 


4.21 


3.84 


2.05 


5= Highest 












1= Lowest 
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Project ALERT 6 Month DAVIS Post-Survey Results 



1 Course Name 


Using a 

Computer/More 

Confident 


1 Course Name 


Using a 

Computer/More 

Confident 


Apprentice Prep 


4 


L Apprentice Prep 


5 


[ Apprentice Prep 


4 


Apprentice Prep 


5 


Apprentice Prep 


2 


1 Apprentice Prep 


5 


f Apprentice Prep 


4 


j Apprentice Prep 


5 


I Apprentice Prep 


3 


1 Apprentice Prep 


5 


Apprentice Prep 


5 


I Apprentice Prep 


3 


1 Apprentice Prep 


5 


f Apprentice Prep 


5 


| Apprentice Prep 


4 


1 Apprentice Prep 


4 


Apprentice Prep 


4 


Apprentice Prep 


4 


I Apprentice Prep 


4 


[Average Rating: 


4.21 


15= Highest 




M= Lowest 
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ugust, 1998 



121 



Learner Expectation Summary 

Please complete this form up to the dotted line when you begin a course. 

Name: Date: 

Course name: Class time/days: 

Job title: □ Male □ Female 

English spoken at home? □ Yes □ No If no, what language is spoken at home? 



(Check the area in which you are most interested.) 

□ Math Improvement □ Reading/ Writing Improvement 

□ Communication skills improvement □ Test-taking skills 

□ Computer skills □ Other (What?) 



Specifically, what would you like to improve? 



Please complete the bottom of this form when you have completed a course. 

Did you learn what you expected to? □ Yes □ Almost □ Some □ No 



If not, why? 



□ Absence 

□ Didn’t understand the work 

□ Time schedule/ work and school 

□ Other 



Would you be interested in other courses? □ Yes □ No 
If yes, what would you like to learn next? 



learexsu 
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Absence 














Slow learner, good teacher 














Not enough time 


1 want/need the next more 
advanced course 


Did you learn 
what you 
expected to? 
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Specifically, What Would you Like to 
Improve 
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1 would like to be computer literate 


Learn more about the computer 


Understanding more about computers 


A general knowledge of computers 


Knowing how to buy a computer - keep up 
w/ daughter who is interested in computers 




Math, Reading & Computer skills 




How to turn it on 




Just learn 


Learn how to work the computer-operating 
Windows & write papers on the computers 


Completely computer literate 


Reading and computers 


Computer skills and the basic functions 


Computer knowledge 


My understanding about computers 


My ability to communicate with others 


Formatting, spread sheets, editing, and 
making files 
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LEARNER EXPECTATION SUMMARY 



If not Why 








































Did you learn 
what you 
expected to? 




Yes 




































Specifically, What Would you Like to 
Improve 




Knowledge & skill of computers 


Computers skills 


Understanding of computers 


Improve basic functions of a computer 


Operating a computer, inc. finding 
information & maneuvering 


To learn the in’s and outs of a computer 


Knowledge & ability to comprehend & 
operate phases of computer 


Computer skills 


All the above skills 


1 would like to improve my communication 
skills 


1 would like to learn to work with computers 


1 would like to improve my algebra and math 
skills and computer skills. 


Basic computer knowledge 






Math Improvement 




Communication 
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1 think a computer. Pretty 
good but I'm a sarcastic 
person & 1 like it 






Did you learn 
what you 
expected to? 




















Yes 


Yes 


Some 


Yes 


Yes 


Almost 






Specifically, What Would you Like to 
Improve 




Spread Sheets 




MAth problem solving interacting with others 




Communication skills with co workers 


Communication skills, and reading/writing 
improvement. 


Computer literacy, communication skills 


Computer skills, Math improvement 


Recognizing and accepting changes 


Basic & advanced knowledge of skills in 
computers & design 


Math 


Learn more about how classmates resolve 
their problems 


Communication skills 


Math computations, basic computer skills, & 
writing improvement 


Improve myself 


Public speaking w / Ig. Grps & ways to better 
resolve prob. In a professor Manner 
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Absence 




































Did you learn 
what you 
expected to? 












Yes 


Yes 


Almost 




Yes 






Yes 


Yes 


Yes 


Yes 




Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Specifically, What Would you Like to 
improve 




Relate job ideas & concepts to employer 
and manager 


My math skills & test-taking skills 


Upon completion of this course, 1 'd like to 
become a better listener 


Listening better and expressing myself 
better 


Learn how to communicate w / people who l 
have a difference with & that 1 don't 




Communication skills, and Computer skills 




To listen clearly & communicate better 




1 would like to communicate with people 
better 




How to deal with snobbish people-those 
who think they are better than others 


Being better at communicating by talking 
and listening 




My Computer and Algebra skills 


1 would like to complete my associates in 
Applied Science in Information Systems 


Computer skills 






Communication & Writing skills 


Reading and writing 


Basic overall improvement 




Course Name 




IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


IPC Axle 


Course 

Number 




20912 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20913 


20914 


20914 


20914 


20914 


20914 


20914 


20914 


20914 


20914 



CO 

vH 



-erJc 



LEARNER EXPECTATION SUMMARY 



co 




>. 

JC 

5 

++ 

O 

c 



CD is 

2 © 
fc -C 

Q $ 



-O 

O 



cd 

</> 

3 

3 

cd 

CD 



a> 

O 05 

a> *| 
<0 .E 

-Q CD 

< -t 



■D 

C 

CD 



05 

■D 

cd 

.c 

o 

to 

cd 

E 



■o 

c 

JD 

to 

i— 

cd 

■o 

c 

3 

O 

c 

■D 

Q 



■D .bi, 
C ^ 

cd a> 
£ 
O O 
> 05 

< C 
© C 

-D © 
© © 

o °s 

w o 
a> o 

il 

h - co 



fc <£ 

« o 

0) 

— 3-0 
3 0© 

© > t) 

■o 2 Q- 
o 



Q 5 



CO 

CD 

> 



CO 

CD 

> 



O 

Z 



co 

o 

E 

< 



a> 

E 

o 

CO 



CD 

E 

o 

CO 



co 

CD 



CO 

CD 

> 



a> 

-x 



3- 

1 

2 
3 
O 

5 

ts 

x 

5 



<0 

o a 

o 8 

03 Q. 

Q- c 

<0 i 



CO 

© ^ 

£ CD 

c *? 

0 O 

2 c 

cd <d 

CL SZ 
CD $ 

1 § 

8 



CD 



c 

13 _ 

1 § 

O a) 

o 55 

2 % 

5 

5 % 



CD 

£ 

■O 

C 

CD 

CO 

u. 

CD 



O 

a> 

E 

a> 

■g 

’co 

c 

o 

o 

E 

o 

E 

CD 

jO 



■D 

C 

CD 

(0 

E 

CD 

i— 

05 

O 



O 

2 

O 



05 

c 

TD 

C 

CD CO 

-«— < r~ 

2 o 

CD *X 

c C 

=3 2 



CO 

k_ 

a) 

3 

Q_ 

E 

o 

O 



a> 

3 

Cl 

E 

o 

o 

c 

c 

o 

2 

c 

CD 

'C 

o 

"O 

c 

CD 



CD 

05 

CD 

3 

05 

C 

CD 



CD 



CO 

a> 

05 

c 

CD 

05 

C 



c 

CD 

"O 



CD 



CD 

CL 

CO 

i_ 

05 

05 

-O 

CD 

CD 

E 

o 

o 

a) 

CD 



CO 

k. 

Q5 

3 

Q_ 

E 

o 

o 

>% 

E 



CO 

o 

‘to £ 

£ cl 

g E 
E 8 
e 'i 

CD > 
3 2 
CL X 

E fl 

Si 

* § 

05 O 



.E 05 

o 

8 e 

E ? 

CD ~ 

5 © 

o 



CD 



Q_ oC 



05 

C 

E 

E 

CD 

l_ 

05 

O 

k_ 

CL 

O 

2 

O 



co 

2 

3 

Cl 

E 

o 

o 

■O 

c 

CD 

£ 

t6 



0) 

E 

CO 

z 

0) 

(0 

k. 

3 

o 

o 



0) 



O 

CL 



0) 

o 

CL 



o 

2 

O 

0) 

i_ 

CL 



O 

2 

O 

Q) 

k_ 

CL 



O 

2 

a 

0) 

k_ 

0. 



o 

2 

O 

a) 

u. 

CL 



o 

2 

O 

0) 

CL 



o 

2 

O 

0) 

CL 



O 

2 

O 

05 

i_ 

CL 



O 

2 

O 

05 

CL 



O 

2 

O 

05 

CL 



0) o 

« X 
3 E 
O 3 
O Z 



05 

O 

CM 



05 

O 

CM 



CO 

o 

o 

CM 



CO 

O 

O 

CM 



CO 

o 

o 

CM 



CO 

o 

o 

CM 



Tj- 

O 

O 

CM 



O 

O 

CM 



O 

O 

CM 



O 

O 

CM 



O 

O 

CM 



O 

o 

CM 




CO 

CO 



LEARNER EXPECTATION SUMMARY 



If not Why 








Time schedule/work and 
school 












Want longer and more from 
the class 




Time schedule/Work and 
school 




















1 thought I'd be using a CNC 
machine 








Did you learn 
what you 
expected to? 




Yes 


Almost 


No 






Yes 


Yes 


Yes 


Some 


Yes 


Some 
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Specifically, What Would you Like to 
Improve 






















Would like to be more computer literate 


Math, Reading & Computer skills 




Math 






Math.computers 




Computer usage, reading and writing 




All of the above 


Test-taking skills 


Computer & Communication skills 




Math, Reading, Writing, Computer, & 
Communication skills 
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LEARNER EXPECTATION SUMMARY 



If not Why 




I've always have had trouble 
with percents and negatives, 
pos. 




Did you learn 
what you 
expected to? 




Almost 


Yes 


Specifically, What Would you Like to 
Improve 




Algebra, Geometry, Percents, and decimals 


1 need more computer knowledge 
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Project ALERT 

Year Two Report, January 1997 

Larry Mikulecky and Paul Lloyd 

Overview 

The evaluators were impressed by the amount and quality of curriculum developed, the level of 
cooperation with partner sites, the number of courses conducted and the number of learners 
served. Though some progress has occurred with the analysis of learner data for impact research, 
we have concerns about lack of progress in two areas strongly put forth in the grant proposal. 
These are impact upon productivity and a structured analysis of learner gains which can be 
compared to control group indicators. Dissemination so far has been good, but we urge the project 
team to take even more action on dissemination and diffusion early in Year 3. 

The detailed comments below report on the project goal by goal, and expand on the issues 
mentioned above. 



Goal 1. To design, develop, and implement innovative workplace literacy programs that are 
tailored to the organization, and the skills and cultural backgrounds of participants. 

This goal is divided into the following objectives: 



Objective LI . To conduct a comprehensive pre-design analysis of jobs, employees, and 

organizations to gather data requisite to the design of effective literacy training. 

This has been done thoroughly at all sites with interview forms for employees and supervisors, 
establishing the overall educational needs at those sites. The level of detail in these interviews at 
some sites (especially in relation to actual job materials and tasks) may not be sufficient at some of 
the more recent sites since curriculum developed for the communications, SPC, and technical 
courses seems to lack much workplace-specific emphasis. For example, the first Davis pre-SPC 
class covered fractions, although it was unclear that they were required to operate SPC at Davis, 
and nearly all the communications course dealt with general communications situations instead of a 
mix of general and workplace-specific situations. 




Objective 1.2 . To collaboratively design and develop workplace literacy programs that are 
responsive to the unique needs of the organizations, jobs, and employees. 

During Year 2, the project has developed several more courses for its partner sites, producing an 
impressive array of offerings: 

• Davis Tool: Pre- Apprentice Prep (previously DAT Prep), 

• Davis Tool: Pre-SPC 1 and 2, 

• City Management: Commercial Driver’s License Prep, 

• City Management: Effective Communication, 

• City Management: Technology in the Workplace, 

• Detroit Axle: Effective Communication Skills on Computer, 
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• Detroit Axle: Interpersonal Communication and Problem-Solving Skills, 

• Detroit Axle: Pre-CNC Operator Preparation (recently renamed Math for 

Machine Operators: From Whole Numbers to Algebra). 

In addition, at the UAW-Chrysler Wayne Center, computer-based learning packages (PLATO and 
JSEP) are being used with a number of small-business partners. The sheer number of these 
courses is impressive and indicates an astounding amount of effort on the parts of all concerned. In 
the rush to get courses up and running, links to job-specific usage (in PLATO, JSEP, and some of 
the more recently developed courses) appear to have been given short shrift. As these courses are 
refined during second and third teachings, they need additional custom-designed materials and 
suggestions to instructors about how to make links to workplace and home use. Detailed evaluator 
comments on the curriculum for SPC Prep, Effective Communication Skills on Computer, 
Interpersonal Communication and Problem-Solving Skills, Technology in the Workplace, and 
Math for Machine Operators are attached to this report as appendices. 



Objective 1.3. To design and develop a range of culturally sensitive instructional methods and 
materials that are both technology and non-technology oriented. 

The evaluators note that this objective has been dropped because interviews with both employees 
and managers indicated that cultural sensitivity was not an issue at any of the partner sites. The 
evaluators’ own observations confirm this: classes were integrated and instructors were ethnically 
diverse, and nobody interviewed by us mentioned this matter as a problem 



Ob jective 1.4 . To design resource-rich classrooms that support traditional and technology- 
based instruction. 

The classrooms at Davis and City Management which were observed previously meet the criteria 
set out in the proposal, and the recent visits to Detroit Axle and the Wayne Center showed them to 
be well-equipped with computers and suitable for the types of instruction taking place in them. 
Some additional materials allowing links to workplace and home may be needed at the Wayne 
Center, where generic PLATO programs dominate existing instruction. Multimedia links to the 
Detroit Axle workplace are an impressive addition to the materials available at that site and a 
cutting-edge example of how to integrate generic instruction with workplace examples. 



Objective 1.5. To recruit p>articipants, implement the site-specific programs and provide for 
reinforcement and transfer of training to the workplace. 

Year 2 has produced a most impressive list of courses (see Objective 1.2 above), and of contacts 
with new companies through the Wayne Center. The project has now taught 10 courses in nearly 
30 sections. However, the changing situations at Davis (internal problems) and at City 
Management (a shrinking pool of students) has produced some recruiting problems. Therefore the 
project needs to be looking for positive changes in workplace situations and consequent new 
opportunities. Discussions with project leaders indicated a strong awareness of this and that plans 
were in place or being considered. Taking advantage of the new Empowerment Zone initiative 
appears to offer productive new opportunities and the evaluators recognize and encourage 
continued efforts in this direction. 

The evaluators were most impressed during their site visit with the atmosphere at the meeting of the 
Detroit Axle Local Joint Training Committee. Rather than vague excuses for why things were 



failing to happen, there was unusually high and detailed cooperation between the project team, 
unions and management on removing impediments to the success of the program. Solutions being 
developed for such problems as department vs. plant seniority and planned access to courses may 
provide ground-breaking guidance to others developing workplace literacy programs in such 
settings. 

One aspect of this objective that is not much in evidence yet is “transfer of training impact to the 
workplace”. The evaluators need more detailed information about what kinds of evidence are being 
gathered to find out if this transfer is occurring. It is a concern that insufficient information is being 
gathered and that opportunities to gather useful information may be missed. 



Goal 2 . To conduct project evaluation and program impact research. 

The evaluators have concerns about progress in various areas connected with this goal. These are 
discussed under the headings below. 

Control Group 

An important evaluation question is: “Would workers not in classes have learned on the job as 
much as other workers learned through classes?” The original project proposal addressed this 
important question through the consistent use of control groups. Changes within the workplaces 
and on-going discussions about evaluation goals have led to some restructuring of the evaluation 
design. For example, not every class is now designed with a control group. 

However, gains made by learners are more convincing if a control group is also being monitored. 
Where classes have waiting lists or sections to be held later, these are natural sources of control 
subjects who can take pre/post measures. In fact, a post measure for the control group could serve 
double duty by being the pre measure once they are able to take a class, thereby cutting the 
additional data gathering efforts by half. We urge the project team to pursue more actively the 
possibilities for control groups, particularly as the randomized treatment research design will not 
now be implemented as originally proposed. It is important that, as part of the fulfillment of its 
stated goals, this project obtains some comparative data on workers who have attended class and 
others who have not, in order to justify the expenditures on course development and 
implementation. It seems clear to the evaluators that gathering such information is possible. Losing 
this opportunity, especially since the promise of gathering such information helped secure the 
funding for the project, would be a shame. 

Data-base Model 

The electronic data-base being developed by the project has the potential to serve as a model of 
data- gathering and analysis for other workplace literacy projects. However, to date, its role has not 
been entirely clear. The evaluators would like to know how the data-base is being used now and 
how it will be used during Year 3. This could be accomplished by providing general descriptions 
of the contents of the data-base and 5-6 examples of how it is being used. In addition, we would 
like to see summaries of the needs assessment and pre/post data gathered for all classes held in 
1996. 

Project Evaluation 

The original program design called for extensive evaluation results by the end of Year 2, including 
6-month follow-up impact data Because of the slow start to the project, evaluation data has been 
available so far ont>nly a few classes, since most began during the second half of Year 2. To date, 
the evaluators have seen complete data (including 6-month follow-up data) on no classes. Data on 
student learning has been mailed to the evaluators for 3 or 4 classes, and a handout providing a 
brief overview of results from additional classes was shared during the October site visit. These 
initial indicators of program impact appear positive. 



Further analysis of program impact is still called for under the project’s evaluation plan. Among the 
areas in which the evaluators need to see information are: 

• Impact on productivity, as indicated by a variety of hard measures and self-report indices. 

• Specific links between pre/post evaluation data and the varying goals of different classes. 

For example, in connection with the latter area, the Technology in the Workplace course and the 
Effective Communication course both have multiple goals. It is sometimes difficult to determine the 
relationship between the goals and the evaluation data being gathered. During Year 3, further 
clarification of these connections will be important 

Extended Impact Research 

This project has ambitiously attempted to go beyond simply evaluating the impact of instruction. A 
model is being constructed to elucidate the inter-relationships between such factors as literacy 
context, target audience, job process, and instructional environment. A theoretical discussion of 
these inter-relationships has been produced and disseminated. During Year 3, the evaluators look 
forward to seeing how this theoretical model is applied to the classes conducted by the project 

Davis Tool Follow-up Survey 

This is being conducted using a questionnaire for employees, plus an interview with those who 
indicate on the questionnaire that they are willing to do this supplementary interview. The fact that 
the questionnaire and interview address several issues not directly related to DAT-Prep suggests 
that this is a general tool for all sites and will be used again. Therefore, it seems appropriate to 
comment on its format. The interview in general is good, but it could be improved by seeking 
specific instances of worker application of what has been learned by asking “Can you give me 
some examples of that?” The questionnaire and interview include both formative and summative 
questions, so perhaps this division could be used as a basis for analyzing the results. It is hoped 
that this analysis will throw light on the research questions related to impact listed above. 

Impact Research 

Though the project has met and exceeded many of its objectives, the evaluators are concerned 
about progress toward meeting the impact research goals documented in the original proposal and 
refined during Year 1 of the project. It appears that a large amount of data has been gathered on 
partner sites and individual learners, but very little has been done yet to analyze this data and 
answer such research questions as: 

• What effect did context have on course design and results? 

• What gains did learners make in areas related to class objectives? 

• What effect did class attendance have on job performance? 

For example, to answer this last question, measures that could be used include CDL and DAT test 
success, and indicators of work attendance, improved quality of work and better communication 
on the job. Data on elements such as these last two could be gathered by systematic qualitative 
interviews with workers and their supervisors, and perhaps also interviews with managers on 
whether project courses made attainment of ISO 9000 certification easier. Anchored rating scales 
could also be developed in relation to these elements. 

«» 

Cost/Benefit Analysis 

Early in Year 2, the project team and the evaluators developed a methodology for assigning costs 
of personnel and materials among the project’s stated goals. Now this method needs to be applied 
to both Y ear 1 and Year 2 in order to update the allocation of time and resources. Also a start needs 



to be made on matching these cost categories with program deliverables and benefits (e.g., learners 
served, course materials published and disseminated, guidance for other practitioners), so that a 
complete analysis of costs v. benefits can be produced, as stated in the proposed. 




Goal 3 . To disseminate the program, program products, and research findings. 

Some dissemination of the project occurred during Year 2— largely through conference 
presentations. Also meeting minutes for late October indicate a serious and concerted approach to 
the dissemination process. As Year 3 begins, we encourage this much more extensive 
dissemination of lessons learned from the project so that other practitioners can learn from this 
team’s experience. In addition to more extensive conference presentation, this should include 
submitting conference papers to the ERIC system, submitting curriculum guides to the ERIC 
system for use by others, and writing up project procedures for recruitment and liaison, for 
research instrument development and data-gathering, and for developing and implementing 
curriculum. At a minimum, these write-ups should be submitted to the ERIC system to expand 
availability. It is the opinion of the evaluators that much of what has been learned by the project 
team will also be acceptable for publication in professional journals. On the way to producing the 
wide variety of courses delivered during Y ear 2, the personnel of this project have learned a great 
deal about how to manage workplace literacy programs in complex workplaces where subtle 
labor/management concerns must be attended to. We know of no other project which has 
accomplished as much in such a wide variety of settings. This, however, only makes it ail the more 
imperative that the lessons learned and the benefits derived from this extensive three-year project 
not be limited to the few hundred students who are served in the Detroit area 

The evaluators suggest that the team put together a “Lessons Learned” summary of about 20 pages, 
outlining the main points arising out of the two years so far, with 2-3 pages from each aspect of the 
project. The structure of this document could be based on the templates developed at the Retreat. 
The authorship of this document could be divided up according to expertise on something like the 
following basis: 

Project management — Dale Brandenburg 

Site recruitment and liaison — Hal Stack 

Learner recruitment and liaison — Irene Sinclair 

Teacher recruitment and training — Irene Sinclair 

Needs assessment — Rita Richey 

Impact research: instruments and analysis — Rita Richey 

Data-gathering on sites and learners — Rita Richey, Nancy Ruetz 

Developing curriculum (whole language) — Nancy Ruetz, Mary Jarvis 

Developing curriculum (multimedia) — Gary Powell, Nancy Copeland 

Implementing curriculum — Irene Sinclair 

This summary could then serve as an outline for each author to use when writing full documents 
on each aspect listed above. These guides should be written for other practitioners— your 
counterparts at another program starting up, and be user-friendly— e.g., including short summaries 
of do’s and don’t’£ for those other practitioners. 

The documents described above need to be disseminated widely to the workplace literacy 
community, through more conference presentations and through journal articles. In addition, we 
urge you to send project documents and curriculum materials to the ERIC system and possibly to 
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set up a WWW site to disseminate materials. However, there is one small caveat: some curriculum 
material is derivative of existing published curriculum and a bit too generic. Acknowledging these 
sources and going a bit more beyond them by adding more connections to workplaces can add to 
the value of the materials you share with the rest of the professional community. (See the review of 
the Axle Communication course in the Appendix.) 



Goal 4 . To promote program institutionalization and diffusion. 

One great strength of this project is the current preparation for its continuation beyond the three- 
year period of federal funding. The recent All-Partners’ Meeting began the necessary raising of 
awareness of the need to plan for continuation among the project’s business partners. Also the 
existence of the Empowerment Zone project should provide an important opportunity for a smooth 
transition to a fourth year of cooperation between Wayne State University and local businesses. 

The efforts being made in these areas are timely and indicative of foresight and good planning. The 
evaluators also observed efforts being made at the Axle planning meeting for courses and course 
ideas to be incorporated into existing Tech Prep programs sponsored by the employer and union. 
The ideas seemed to be well received and discussion of concrete possibilities occurred. This fine 
attention to detail and opportunity on the part of the project director and other project personnel is 
to be applauded. The positive reception of these transition ideas by business and union partners 
reflects the value they place upon what the project has developed and the high regard they have for 
project personnel. It was a pleasure for the evaluators to see educators at the planning meeting 
being treated as full partners rather than simple service providers. We recognize that such status is 
the result of trust being developed through a history of successfully working together to overcome 
the dozens of obstacles which can impede successful program development. 



Summary 

The evaluators have been very impressed by the progress of Project ALERT during Year 2, after 
its slow start in Year 1. Development and implementation of curriculum have been accomplished at 
a high level, clearly fulfilling Goal 1 of the project proposal. Also Goal 4, program 
institutionalization and diffusion, is proceeding in a very satisfactory manner. Our concerns at this 
stage are concentrated on Goals 2 and 3, and we recommend that the following be given top 
priority in Year 3: 

• impact research : as a matter of urgency, a control group needs to be found for at least one up- 

coming class (and preferably several classes)— to validate the curriculum approaches of the 
project; 

• i mpact research: the project data-base needs to be used to assess learner gains in completed 

classes, particularly in the area of job performance; 

• cost/benefit analysis : costs of personnel and materials need to be matched with program 

deliverables and benefits, to produce an analysis of costs v. benefits; 

• dissemination : project procedures and lessons-leamed need to be written down and explained, 

so that other practitioners can learn from the experiences of this project 
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Project ALERT — January 1997 
Comments of external evaluators on curriculum drafts 



SPC PREP 

These two courses, SPC Prep 1 and SPC Prep 2, are very different in their presentation. SPC Prep 
2 seems very well focused on SPC, the need for it and how to do it. However, SPC Prep 1 does 
not address SPC at all until the very end, with calculation of averages. This may present a problem 
of motivation for students of SPC Prep 1. 

It seems to be feasible to streamline these two courses into one, targeted very specifically at the 
statistics needed for SPC. Then the only arithmetic needed would probably be addition and 
subtraction of decimals and their division by whole numbers (most likely with the use of a 
calculator). Fractions may not be needed at all. This issue could be resolved by conducting a task 
analysis of exactly what arithmetic skills are needed to carry out the SPC process at Davis Tool. 



SPC Prep 1 

As mentioned above, this course needs early motivation related to SPC — What is SPC? What does 
it look like in action?— so that workers know why they taking the class. This could be 
accomplished in a variety of ways: e.g., a written description, a shop-floor demonstration, or a 
video of SPC being carried out. Then, workers will be able to see why they are learning certain 
skills in the class and why the course builds up toward the calculation of averages and leads into 
SPC Prep 2. 

Before considering the curriculum in detail, we have a few comments on the overall structure of the 
course. The lesson format that builds from prior knowledge toward practical application and job 
relevance is a good one, and the word list and glossary at the end are good and thorough. 

However, the purposes of the Preview and Review are less clear. Is the Preview used as a 
diagnostic tool before the course to decide where learners have particular weaknesses? The Review 
is very short (only one page) and does not cover the whole course content. It is not clear how this 
is to be used. 

Several lessons in the course are devoted to fractions, which may not be needed for SPC at Davis. 
However, if fractions are used, learners will require a more thorough approach if they are to 
understand a topic that they presumably have failed to grasp in the past The lessons on fractions 
begin with a good introduction via equal parts, but then go straight into common denominators and 
four rules without further explanation or justification. For learners to understand the processes 
related to fractions, they need to be able to refer back to diagrams of equal parts at all stages of their 
development, so that they can understand, for example, why a common denominator is necessary 
in certain situations. 

The sections on decimals and negatives follow a similar pattern, beginning slowly with examples 
and justifications, but then accelerating rapidly into rules for calculation without providing reasons 
for why we do it t^iis way. For instance, adding and subtracting negatives can best be appreciated 
by always using a number line or thermometer, rather than trying to remember a rule with no 
meaning attached to it Similar remarks apply to multiplying and dividing negatives, and the 
rounding of decimals. 




However, a more basic issue is establishing the need for developing workers’ skills in these areas. 
At Davis Tool, are negative numbers used in SPC? Are fractions? Is multiplication of decimals 
needed? Do workers use calculators on the job? Answers to such questions should drive the focus 
of this course, and allow the distinction to be made between need to know material and nice to 
know material. The former means those skills that are necessary for workers who will be carrying 
out SPC on the job-these must be included in the course; the latter means other skills which may 
be of use to workers in their everyday lives— these can be included if there is time after the 
essential skills have been mastered. 



SPC Prep 2 

This course starts with a good introduction to issues of quality, statistics and SPC. If the two 
courses remain separate, such an introduction should be outlined briefly at the start of SPC Prep 1 
to motivate workers and make it clear to them why they are taking the course. 

The general remarks on the effectiveness of the lesson format made above for SPC Prep 1 also 
apply here, but there are two additional features that deserve comment. The readings on quality are 
an interesting inclusion, but it is unclear how these are to be used: are they an integral part of each 
lesson, or an optional extra? The “Daily Report” is an excellent idea, but should ask both how you 
learned and why you need to know. Both questions encourage useful reflection about the learning 
process and encourage self-motivation. 

Finally, we would like to note that the current version of this course is incomplete: some lessons 
are still blank, and pre/post tests are missing. However, this second SPC Prep course is clearly 
targeted at the implementation of SPC and, as remarked earlier, a single course with this structure 
and streamlined to address the SPC needs of Davis Tool might be a more effective way of teaching 
what is currently two courses. 



EFFECTIVE COMMUNICATION SKILLS ON COMPUTER 



This course makes good use of Detroit Axle workplace material and context within the framework 
provided by the “New Reading Disc”. The development team has wisely avoided making much use 
of the existing content material on the disk, which has a strong British flavor in its spelling, 
vocabulary and pronunciation, and so may be confusing to American workers, particularly those 
with weaker English language skills. The “New Reading Disc” also provides some structures that 
may not be very helpful for developing communication skills; therefore the program should be 
used with caution and careful thought as to the purpose of particular exercises. However, most of 
the structures in the program do appear to be useful, especially the scaffolding provided to reluctant 
writers in the form of ready-made sentences that they can manipulate into a story or argument. For 
example, when workers are writing a job description, it will be very helpful to them to have 
models of what this might look like. 

The course guide claims that the four areas of Computing, Work process, Reading and Writing are 
covered in separate lessons. This does not seem to happen and is in any case undesirable. The 
visible mix of these skill areas is good and very appropriate. 

We note that “Planning and Scheduling” listed in the Introduction is missing from the version of 
the guide that we have seen. 
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INTERPERSONAL COMMUNICATION 
& PROBLEM-SOLVING SKILLS 



The lessons are clearly organized and easy to follow. The introduction and explanations to teachers 
are straightforward and appropriate for a communication course. There are many good activities 
which will need much more practice. Fortunately, there are many other contrived activities which 
could be dropped to provide more practice and transfer time for the stronger parts of the course. 

For details, see the notes below. 

Lessons 1-3 

The opening activity is a good way to establish rapport and break the ice. The “setting guidelines” 
activity of Session 2 is also a useful and real activity worthy of modeling communication. It would 
be nice to follow this up with more realistic, job-related activities. Maze mania and visualizing a 
secret garden seem a bit contrived and do not seem to push forward the learning of communication 
skills as well as they might. We recommend that you develop more work-related activities. Did you 
find the heavy use of stress reduction techniques to be necessary? The goal of the “human camera” 
exercise seems useful, but it seems a little too game-like. We suggest a video of a work-related 
incident to accomplish the same goal of demonstrating different perspectives. 

Lessons 4-6 

Do you still need "getting to know you" exercises by Session 4? The listing of communications 
needs on the job is good; we suggest starting this in an earlier session. Pretending to be animals 
and making animal sounds seems a bit too contrived when there are so many other more important 
uses for the severely limited time. The emphasis on defensive behavior and the use of video in 
Session 5 is very important. It would be useful to conclude the session with a link to the 
workplace, so that transfer of learning is more likely. It would be useful to add some team 
examples instead of the manager/employee example. Could the telephone example in Session 6 be 
made more work connected— perhaps describing the size of a room to be painted or how much 
carpeting might be needed to cover a floor? The "real experience" forms are a good idea. How did 
they work? Self rating and parent communication rating are also good ideas. 

Lessons 7-9 

The active listening skills are good. Again, we think these could occur earlier in the course if some 
of the more contrived activities were dropped (e.g., animal sounds). The “body search” activity 
seems a bit better— especially if the link to workplace communication can be made even clearer. 

The “sender-receiver-observer” activity seems like a good way to provide practice for active skills. 
We would like to see this extended, with the teacher and other students providing feedback about 
effective examples of student practice. Also, make more links to the workplace. Social bingo does 
not seem necessary— did the teachers find this much stress reduction required? It does cut into very 
important learning, practice, feedback, and transfer time. This could be a chance to introduce and 
practice the model which would then be elaborated upon in Session 9. 

Lessons 10-12 

Time on the model is well spent, especially if connections to work and family communication can 
be expanded. Video and any realistic application is excellent The “know game” seems contrived. 
We suggest these activities be listed on a separate page and be recommended only if, after 10 
weeks, the group still seems overly stressed. We think this will be unlikely. The problem-solving 
activity seems useful, but the connection with the self-discovery exercise seems weak. You would 
need to spend a good deal more time than a single exercise provides for this to prove useful. The 
goal of “creative solutions” is good, but we do not think the activities are likely to get someone 
there. More brainstorming and evaluating of ideas is likely to be more effective. Research does not 
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show much transfer to real-world applications from the sorts of word exercises on the IQ sheet and 
the "bibble" exercise. 

Lessons 13-15 

This lesson is nearly 2/3 of the way through the course. By now the activities should be much 
more work-connected and less dependent upon contrived games like mimicking each other and 
“name the soap”. The goals are good, but the activities can be improved a great deal. We suggest 
dropping Lessons 13 and 14 and providing more practice on problem-solving and practicing active 
listening skills. Skills are not mastered in a single session. The skills covered earlier are much 
more important than those covered in these lessons. The goal of reaching consensus is the most 
important goal in these three lessons. It should be expanded and be used with several different real 
problems which emerge from the class. If nothing else occurs, plan an event or make decisions 
about ordering food. 

Lessons 16-19 

If the students still need to learn to feel comfortable about being in a group by Lesson 16, 
something has not been working in previous lessons. We recommend that you skip Session 16 and 
go to Session 17 on the "I" messages. We do not think you still need icebreakers this far into the 
course. You should build more links from "I" messages to the workplace and family use. The 
“values clarification” exercises seem out of place given the goals of the course and the need to 
practice and transfer the important skills you are presenting. We think that much of the last 3-4 
sessions should be spent making connections to the personal work and family lives of the students. 
Too little direct class time has been allocated to team communication and the specific 
communication difficulties experienced in the workplace. 
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TECHNOLOGY IN THE WORKPLACE 



Front Material 

The instructor's material in clear and useful— especially in terms of methods for dealing with 
different sorts of learners and different sorts of group activities. More connections and suggestions 
related to this "front" material should be made in the very well planned daily lesson guides. The 
course objectives are clear cut and seem manageable for the time involved. 

Minimal Literacy Level of Learners 

Saying the course is not for non-readers is of some use, but you should go further. What minimum 
do you really mean (i.e. 4th grade reading ability? 8th grade? 10th grade?). You should provide 
some early activity to help the instructor find learners who are not likely to do well in the course 
and then suggest what might be done (e.g., counsel them out, or let them use the computer 
independently to learn to read better). 

Daily Lessons 

The daily lessons appear to be very well planned. Activating prior knowledge and making 
connections is a sound beginning for lessons. The lessons are clear and simple enough to provide 
success for nearly all learners. The use of games to practice computer skills is a good idea in the 
early lessons, and also allows for early success. We suggest that you allow the option of doing 
worksheets independently or in pairs. 

The lessons are logical and develop in a reasonable way. When you get to word-processing, we 
suggest you have the learners start to create a document linked to their job or family needs. This 
can be used throughout later lessons to practice with spell checker and other tasks you wish to have 
students use the computer for. Perhaps the journal activity on page 70 could be started in a 
rudimentary form even earlier. The newsletter is a good idea, but might not work if learners do not 
share a good deal in common. In this case, you could create the possibility of a mixture of 
individual and group projects which use the computer. 

Addition to Pre/Post Inventory 

We suggest adding to the pre/post test and inventory some open-ended items which ask learners to 
list the ways in which they have encountered and/or used computers at home and at work. 



MATH FOR MACHINE OPERATORS 



The evaluators viewed this package during their visit of October 1996, and then reviewed it in 
more detail in December 1996. On the first occasion, the program was operated by its developers 
on its usual platform and so no technical problems arose. In the more detailed review, the program 
needed to be loaded from disks (which caused some difficulties that are described below), and the 
evaluators were joined by people with greater knowledge in the field of Instructional Systems 
Technology: Professor Thomas Duffy and a graduate student Chandra Hawley. Their overall 
impression of the package was not so favorable as that of the evaluators at their first viewing. 

This multimedia course package gives a good introduction to the Detroit Axle workplace and its 
inter-connected departments (through the “Getting in Gear” section), and provides mathematics 
instruction and exercises in those areas needed by CNC machine operators. It is obvious that a 
great deal of effort has gone into the design and execution of the package, which goes a long way 
toward fulfilling its objectives. However, there are a number of problems that the evaluators wish 
to comment on, with a view to improving the effectiveness of the package. These are set out below 
under the following headings: 

• interface with other computers and difficulties of loading; 

• general design features that impede program use; 

• design details that interfere with easy use; and 

• a suggested remedy. 



Interface 

The program is very difficult to load from its six zip disks. It takes over an hour to load and 
requires such a large amount of memory that special technical arrangements are needed to 
accommodate the program. Both a graduate student in the Instructional Systems Technology 
department and a specialist in the university’s Technology Laboratory spent many hours (5 - 10 
hours each) trying ingenious ways of getting the program to load and run, before finally 
succeeding. Even then, we appeared to have some problems related to loading. In the “Getting in 
Gear” section, the department descriptions were not accessible from the menu screen, but only by 
calling up the files from the main menu. And the fonts and colors we obtained were not as good as 
we had seen in the demonstration at Wayne State. We suggest that the desired fonts are loaded with 
the program. 

All of these problems could be overcome by putting the package on CD-Rom. The cost of burning 
a CD is now less than $10 (much cheaper than six zip disks), so this need not wait until the 
package is in final form. In fact, the equipment needed to bum a CD is itself now in the $500 - 
$2500 range. Having the draft package available as a CD will make its distribution for comment 
much easier, because the memory problems we had in loading will not arise and CD-Rom drives 
are more readily available than zip drives. 

Design: general 

When a user first enters the program, it is important to make a good and user-friendly impression. 
The complicated CNC keyboard is not needed on the opening screens, and would be better omitted 
from them. The testimonies (of Virdell King and Tony Mason) have a very clear and positive 
purpose in persuading workers of the value of the package, but these testimonies are not very clear 
or well-read. They need better sound quality and more expression from the readers, including eye- 
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contact with the viewer. Perhaps they could be re-recorded using a tele-prompter. The later 
narration also needs better sound quality and more expression. It is not expensive to hire a 
professional voice, and would make a much better impression. 

Once the user has moved into the “Getting in Gear” section of the program, there is a need for 
more instructions about when and where to click— many of the workers using the package will 
have had little experience with computers. On the other hand, those with experience may be misled 
by seeing text in blue, into thinking that these are hot links as on the World Wide Web. It would 
help to add headers for Pinion and Ring Gears above the lists of departments on each side of the 
screen; the pictures are not enough of a clue. In addition, it is unclear how one ends a session with 
the package: the Quit button does not exit from the “Getting in Gear” section. We needed to restart 
the whole program to move from one section to the other. 

Within the mathematics section, we have two kinds of comments: on the mechanics of completing 
answers and on the pedagogical approach. It is a little confusing to find yourself moved to the next 
box requiring completion when you press the Enter key for your current answer. Also the next 
screen is activated by completing the bottom right entry only (rather than the full table). 
Pedagogically, it is undesirable for a student to receive the bald response “Wrong”— better would 
be “Try again” or “For help, hit . . .”. And the CNC forms would look more realistic with the 
appropriate workplace header material, which could be customized for each client workplace. 

Design: detail 

Some details of the program’s technical design proved annoying in use. Activation by moving the 
cursor over text (e.g., the toggle between Virdell’s and Tony’s speeches) can happen without the 
user wanting it to; choosing to click on text is preferable. Some displays were distracting: the 
numbers among the words in “Welcome to Decimals”, the multi-colored numerals (not appropriate 
for adults), and the shadow letters behind some text in a different font. 

In the “Getting in Gear” section, the text was difficult to read, because too little appeared on any 
one screen and the lines are so short. The latter problem could be ameliorated by not right 
justifying (to avoid large gaps between words), but it would be better to use a smaller font so that 
more text will fit on a screen— or even better to narrate this material, as the package is not intended 
to teach reading. 

In addition to these design issues, there are many unintentional and proofing errors. We give only 
a few examples here. They include grammar (e.g, “you” for “your”), spelling (e.g, “labled” for “ 
labeled”), and formatting (e.g, inconsistent spacing after bullets). 

Suggested remedy 

It is clear from the comments above that the package needs a very careful proofing and design 
critique before it will be ready for general distribution. Therefore we suggest that the project hire 
somebody to advise in detail on desirable changes and maybe also to make those changes. Two 
graduate students at Indiana University are willing to do this under Tom Duffy’s direction. They 
are Chandra Hawley and Jason Orvill, who can be contacted at (812) 331-2654 or by e-mail at 
chawley@indiana.edu. 
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Project ALERT 

Report on Draft Curricula, March 1996 

Larry Mikulecky and Paul Lloyd 

Drafts of two curriculum packages have been sent to us. What follows 
below is feedback organized into areas of strength followed by suggestions 
for further strengthening the units for eventual dissemination and some 
questions of clarification. 



DAT Preparation 



STRENGTHS 



The manual has clear directions for the instructor. 

Materials and activities link to both the DAT test and to a variety of in-plant jobs (e.g., Inspector, 
Die Setter, South Line Operator). 

The Lesson structure incorporates a solid understanding of current research on learning. Each 
lesson activates learner prior knowledge, provides instructor modeling, provides practice in a 
variety of formats (single, pairs, groups, whole class), provides a link to workplace applications, 
and offers learners an opportunity to comment upon and evaluate each lesson. 

Reading comprehension questions provide a good mix of question types (vocabulary, text implicit 
questions and text explicit questions). 

The technical vocabulary and writing lessons demonstrate a clear link to the Davis workplace and 
show evidence of literacy task analysis. 

The mathematics section is introduced with the sound philosophy and purpose of linking learning 
to practical applications. This is then followed in the lessons, as far as the requirements of the DAT 
test allow. 

The first fractions lesson provides an excellent start on understanding how fractions work. 
However this is not visibly linked to the rule-based practice of the four operations, which is where 
many learners get lost. 



SUGGESTIONS 



The manual could benefit from a 3-4 page introduction which would describe a bit about DAVIS 
and the DAT. This section could comment upon how the materials could be used or modified to 
help prepare for other tests. 

In addition, this introduction section should provide a brief explanation of how to use some 
activities you suggest (i.e. Jigsaw groups, say something, and reciprocal questioning) 

The addition of some DAVIS or DAT-like examples in the Workplace application section of the 
test-taking activities would strengthen this section. 



Most times you indicate the sources of exercise materials. This should occur in a consistent 
manner. Probably at the top or bottom of each page you use would be best. 

"Tips for Test-takers" list is quite long. You could break it up with some sub-headings like: At 
Home, Getting Ready in the Testing Room, and During the Test. 

The Writing Activities section of the manual sometimes does not provide samples of the student 
material mentioned on the guide sheet. These should be added when needed. 

The index for Writing Activities gives the impression that many more lessons are being developed. 
If this is not so, this page needs to be changed. 

The index for the mathematics section also suggests gaps to be filled, and some problem sets 
appear to be missing. 

The section on place value would work better as an introduction to the section on decimals, which 
extends the idea of place value. 

The first lesson on percentages concentrates on % of amount, but the exercise that follows includes 
all three types of percent problem. This mix may be confusing to learners. Also, will they really be 
doing this by long division or do they all have access to calculators? 

In the section on finding unknowns, use of examples where the unknown is a digit within a 
number may cause confusion with multiplication of a number and an unknown (e.g., compare 
3m46 with 3m). Such examples would be better avoided for this reason - and because they rarely 
arise in real life. 




Building Effective Communication Skills 



STRENGTHS 



Useful initial statement of whole language philosophy. 

The manual has clear directions for the instructor. 

The Lesson structure incorporates a solid understanding of current research on learning. Each 
lesson activates learner prior knowledge, provides instructor modeling, provides practice in a 
variety of formats (single, pairs, groups, whole class), provides a link to workplace applications, 
and offers learners an opportunity to comment upon and evaluate each lesson. 

Moving toward production of a newsletter allows for incorporation of most effective elements of a 
whole language approach. 

Suggestions for Adult Learners and Tips for Teaching are useful. Consider adding these to DAT 
curriculum. 

Glossary of suggested activities is marvelous— practically a textbook in itself. 

Course goals and activities are well conceived though perhaps too ambitious for just 30 hours of 
instruction. Most goals will take more than a single lesson to accomplish. 

"Plepto cheese" example is great 
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SUGGESTIONS 



The Learning Styles suggestions are useful. They could be strengthened with some suggestions to 
instructors about how to identify a learner’s particular learning strengths. These need not be 
extensive. 

There is a bit of contradiction is saying there are no prerequisites, but that the course is not a 
beginning reading and writing class. You might consider saying that the course is designed for a 
minimum of intermediate level literate (i.e. 5-6th grade plus) in the prerequisites section. 

The Four year olds at the Fire Station selection is about high school difficulty level— primarily due 
to sentence length. It is comparable to a newspaper article. If the majority of students is below this 
level, an easier selection for the beginning of the course would be better. Your suggestions for oral 
reading help, but you want to avoid early frustration with the materials you provide. The Maslow 
piece is a bit easier to read, but still upper middle-school level. 

You could introduce the idea of interviewing for a newsletter during the Interview lesson. A good 
activity might be interviewing a fellow classmate with the goal of writing a brief newsletter profile. 

To incorporate the Whole Language approach you introduce this unit with, you might consider 
building in more student choice in activities. This could be linked to prepairing the newsletter. 
Students could create topics for interviews, decide which materials should be skimmed and 
scanned, what activities require notes, etc. 

Many of the objectives are unlikely to be accomplished in a single lesson. You address this 
somewhat by suggesting that teachers may wish to select goals and customize instruction. A useful 
addition would be a suggestion of additional materials an instructor could use if additional lessons 
were needed. As a beginning, these could be specific chapters from your reference list. 

If computers and word-processors are available, you could consider introducing the idea of the 
newsletter very early and having many of the lessons lead to brief stories which could be written, 
spell-checked, and later included in a newsletter. 



3 



155 



Project ALERT 

Report on Multimedia Development, March 1996 

Larry Mikulecky and Paul Lloyd 

The multimedia package is being designed for the Chrysler Detroit Axle 
plant, to assist workers in the transition to higher technology particularly 
the use of machines with Computer Numerical Control (CNC). The 
package will include: 

1. 10-12 math modules focusing on the use of decimals in CNC 
applications and including a simulated CNC panel, 

2. a tutorial on gear and pinion theory at just sufficient depth to provide 
workers with an idea of what the machines they operate are doing, 

3. a glossary of key terms accessible from the other units by hot words, 

4. a description of the process flow through the departments involved 
in production of gears and pinions, designed to increase workers’ 
awareness of the whole process. 

At the visit in late March 1996, the only part of the package seen by the 
evaluators was a partially- completed version of the first math module 
(introduction to decimals) and some of the overall structure of the 
package. The evaluators were very concerned about the slow progress on 
the package, with only one of 10 modules started, and about the 
instructional format in that module, which did not make much use of the 
flexibility possible in a multimedia package. 

Based on the sample seen, we have the following more detailed comments. 

Strengths 

1. Established need 

Based on a task analysis of the skills workers need to operate CNC 
machines, a design matrix has been developed that clearly sets out the 
relation between the necessary mathematical skills and the models and 
methods to be employed in the multimedia design. The evaluators’ 
interviews with Chrysler Detroit Axle employees and managers confirmed 
the nature of this need. The managers, in particular, emphasized that 



training related to the operation of CNC machines was central to the 
plant’s effort. They believed that this multimedia package had great 
potential for assisting the plant to move smoothly toward an increasingly 
CNC-based future. 

2. Program introduction 

The evaluators were impressed with the opening sequence to the 
package, with its engaging title. "Get in Gear”, and the letters rotating as 
gears. The proposed introductions and endorsements from key players 
were also seen as a good idea. 

3. Sample math module 

In the partially-completed version of the first math module (introduction 
to decimals), the evaluators saw as very positive the use of multiple 
approaches to the structure of decimals — the base 10 cube, divisions of 
the number line, and successive place values. 

Suggestions 

1. More flexibility for learners 

The present prototype module appears to be too linear — i.e., the learner 
is locked into a sequence of screens, often several text screens leading 
toward a question item. The developers ought to incorporate the 
possibility of learners testing out of a module or the easier parts of a 
module via diagnostic questioning. For example, the wrong answer to an 
initial easy question on place value could direct the learner to the 
multiple approaches to the structure of decimals, whereas the right 
answer could lead to a more difficult question. In turn, the wrong answer 
to this question could direct the learner to the section on successive 
place values (skipping the approaches to the structure of decimals), while 
the right answer could allow the learner to test out of the module 
completely. 

2. More learner interactivity 

At present, the prototype, module contains sequences of several screens 
which the learner just reads, without carrying out any activity. It is 
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recommended that the learner has an activity to do at least every second 
screen, and whenever possible on every screen. Examples might include: 

a. recall questions related to text just read, to provide learners with 
immediate feedback on understanding: 

b. options to click on to move directly to workplace and/or real-world 
examples of concepts being discussed: 

c. options for learners to suggest workplace and/or real-world 
examples by typing to look for a match with a master-list. 

3. Use of screen colors 

In the prototype module, there were some places where the text did not 
stand out clearly against the background because of insufficient color 
contrast. Research indicates that formats close to print pages work best — 
i.e., dark text and light background. Also, whether type size is large 
enough is a consideration, especially if the package will be used by older 
workers. 

Related to the issue of color contrast is that of consistent use of distinctive 
key colors to signal, for example, links to the glossary. It will enable 
learners to use the package more easily if they can learn these “codes” 
very quickly and become comfortable with them. 

S ummar y 

The multimedia package has the potential to be a very useful instructional 
tool, but very little of that potential has as yet been translated into actuality. 

Its development has a long way to go yet, and the evaluators are concerned 
that it will not be ready in time for its expected implementation at the 
Chrysler Detroit Axle plant. 

Assuming that it can be finished in time, the structural changes suggested 
above will, we believe, enhance its usefulness to Chrysler Detroit Axle plant 
employees. Certainly, the managers interviewed at the plant see this 
package as a key component in their drive toward greater computer- 
controlled operation and look forward to its implementation with great 
anticipation. 
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Project ALERT 

Report on Research Plan, March 1996 

Larry Mikulecky and Paul Lloyd 

The research plan developed for the project appears to be a good one 
and is now about to come into full implementation as the first classes 
are completed. One aspect of the plan already accomplished is the pre- 
design analysis or needs assessment at each worksite. The evaluators 
suggest that the needs assessment model and its applications be 
thoroughly documented as soon as possible, and be prepared for 
submission as journal articles, ERIC documents, and conference papers. 
Because of the wealth of material gathered by the project, it will be 
advisable for the team to brainstorm possible topics and audiences, and 
then prioritize among them to decide where best to channel their 
efforts. 

With regard to the gathering of learner data at Davis Tool, it seems that 
those workers who signed on for the first wave of the DAT preparation 
course have been reluctant to complete a post-test and thus supply the 
project with a control group. The evaluators suggest that these workers 
be paid for their time in completing that post-test, an approach which 
is more likely to be successful and will be less time-consuming than 
repeated persuasion. 
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Project ALERT 

Report on Site Visits, March 1996 

Larry Mikulecky and Paul Lloyd 

DAVIS TOOL 

Interviewees 

Steve Nelson, Director Human Resources 
Mike Wysocki, UAW Plant Chairperson 
Thomas Buvia, program participant 
Israel Mitchell, program participant 
Chauncey Cooper, instructor 

1. Education Needs 

These are seen as a mixture of job preparation (for QS 9000 certification) 
and more general individual educational enhancement. The Human 
Resources Director and the UAW Plant Chairperson tended to emphasize 
the increase in workplace technology and the need for greater worker 
skills to handle that, while the instructor and the participants 
emphasized individual development — “I’m using my mind again” — and 
described widening of horizons to include continuing education and 
possible job changes. All interviewees saw the basic skills of reading and 
math as vital for both job preparation and more general educational 
advancement. 

2. Current classes 

All saw the first classes (DAT prep) as very useful. The learners 
emphasized the topics of math, direct preparation for the DAT test, and 
test-taking skills. One contrasted his positive experience doing the DAT 
test with that of others who had not attended the class: “The class 
exercises helped a lot. Those who didn’t attend had a problem.” This was 
supported by the Human Resources Director, who stated that class 
participants’ scores on the DAT were as good as those of senior 
employees. The instructor mentioned changes in attitudes as participants 
developed understanding of concepts and confidence in themselves. He 
summarized his impression with: “It’s a good program and it’s needed.” 
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More generally, both the Human Resources Director and the UAW Plant 
Chairperson had heard very positive comments from participants, such 
as: “This is a big leap forward for me” and “When are we doing this 
again?” The UAW Chairperson reported an excitement, people with “a 
smile on their face at 4 pm”; he was pleasantly surprised that people 
were sticking with it. 

One issue mentioned by all interviewees was the dropout rate for the 
classes (which was about 50%), but possible reasons for this varied 
widely. The Human Resources Director, one participant and the 
instructor saw it as a problem of mixed ability classes having to cover the 
material together, even when progress was too fast for some and too slow 
for others. The instructor said: “I have to move on in the test 
preparation, but some are left behind." This problem was dealt with to 
some extent by the instructor providing extra help sessions on non-class 
days. The participants judged their fellow workers more harshly: “Some 
people were not ready, were not able to do this, and didn’t belong here” 
and “They don’t have their priorities straight." The UAW Plant 
Chairperson attributed the dropout rate to the fact that the DAT classes 
were off the clock, adding that the next class (pre-SPC) would be on 
work time. (However, he also stated that, speaking personally, he would 
prefer all classes to be off the clock, so that attendance was based on 
“pure enthusiasm.”) 

3. Classroom facilities 

The classroom, which is located above the shop floor, is a former 
storeroom that has been converted by the company into a teaching area. 
All interviewees agreed that the classroom was very suitable, the 
instructor and one learner comparing it favorably with school classrooms 
they knew. Its location right on site was seen as a great advantage. 
However, this also produced its one major disadvantage — the noise and 
vibration from a press operating underneath the classroom. Only one 
possible change to the classroom was mentioned: the instructor would 
like more board space. 
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4. Teaching materials 

Most made little comment on this area. The Human Resources Director 
and one participant mentioned the close match with the DAT. Not 
surprisingly, the instructor had most to say. stating that the materials 
were “adequate and getting better as we go along." He mentioned 
changes of emphasis in the second session (more on verbal analysis and 
spatial concepts, less on fractions), probably amounting to a 30% change. 
He thought the directions to instructors were about right — specific 
enough, but also allowing flexibility to the teacher. 

5. Instructor priorities 

All interviewees (except, of course, the instructor) had high praise for his 
commitment and availability to help those who were having difficulty. One 
participant stated: “He's not in a real hurry to get the heck out of here" 
at the end of class time. The participants were also impressed by his 
! giving out his home telephone number for them to call on Saturdays. The 

Human Resources Director thought that this caring attitude may have 
kept some workers attending the class. The instructor himself saw his 
priorities as evaluating student needs and attitudes, and then evaluating 
skill levels. He also mentioned the importance of feedback from and to 
students: “You need to let them know you care.” 

6. Recruitment 

The main method of recruiting was through meetings and presentations 
to all employees, involving both Wayne State and site personnel. One 
participant also mentioned that the message was passed along by word- 
of-mouth within the plant. The UAW Chairperson saw the "carrot of 
apprenticeship" as a factor. (Passing the DAT test is one of the screens 
for entry to the apprentice program.) However, both participants 
regarded this as secondary: “I would like that, but I'm doing this for 
me.". The instructor saw the DAT link as making a difference initially, but 
that workers' goals shifted later toward more general educational 
advancement. No interviewee questioned the effectiveness of recruitment 
and the Human Resources Director commented that already 20% of the 
hourly workforce had attended the classes. 

O 
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7. Connection between class and workplace 

The DAT classes were seen as having some connection with the 
workplace through the opportunities for advancement that they provided. 
However, both the Human Resources Director and the UAW Chairperson 
saw “the biggest one coming" — the pre-SPC class would allow the 
company to overcome past weaknesses in SPC use. 

The instructor took a wider view of the workplace connection, dealing in 
class with career security and encouraging workers to develop skills 
other than for Davis through other educational institutions. This extended 
to bringing to class community college enrollment information. Both 
participants had picked this up and spoke of their need for more 
education as a preparation for an uncertain future: “Davis may not 
survive — education helps" and "If I need to get a new job. I’ll be better 
prepared.”. 

In summary, all interviewees thought that the program was going well and 
meeting company and employee needs. They very much valued the hands-on 
involvement of the Project ALERT team, particularly that of the Program 
Coordinator. The Human Resources Director commented that “the key 
contact people are keeping them [Davis] in the loop — perhaps too much!” 
This last remark referred, jokingly, to the number of meetings and the 
occasional difficulty of scheduling them. However, the effort to maintain 
contact at all times was clearly appreciated. 



Relation of Davis classes to program objectives under Goal 1 

[To design, develop, and implement innovative workplace literacy programs that are tailored to the 
organization, and the skills and cultural backgrounds of panicipants.J 

Objective 1.1 . To conduct a comprehensive pre-design analysis of jobs, employees, and 

organizations to gather data requisite to the design of effective literacy training. 

There is surprisingly clear consensus among all stakeholders on the nature of employer and 
employee needs and that these needs are being met It is unusual to find such agreement during 
the first round of a workplace literacy program. This consensus is indicative of a well- 
conducted needs analysis. 
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Objective 1.2 . To collaboratively design and develop workplace literacy programs that are 

responsive to the unique needs of the organizations, jobs, and employees. 

Both the DAT and pre-SPC classes are seen as relevant to the workplace and appropriate for 
the workers as individuals. 

Objective 1.3 . To design and develop a range of culturally sensitive instructional methods and 
materials that are both technology and non-technology oriented. 

This does not appear to be an issue at Davis; no comments were made about it and the nature of 
a test-preparation course seems to allow little latitude in curriculum emphasis. 

Objective 1,4 . To design resource-rich classrooms that support traditional and technology-based 
instruction. 

The classroom provided by Davis is adequate to its purpose. The only qualifications to this 
concerned noise level and a desire for more board space. If the program expands to include 
more emphasis on individual learner development, then more learner materials and space for 
their display will be needed. 

Objective 1.5 . To recruit participants, implement the site-specific programs and provide for 
reinforcement and transfer of training to the workplace. 

Recruitment has been very successful, the classes and their instructor have been well received, 
but the extent of transfer to the workplace is as yet unclear (and depend on the results of the 
DAT test). It is worth noting that achieving 20% employee attendance during a first round of 
courses is very high. 
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Project ALERT 

Report on Site Visits, March 1996 

Larry Mikulecky and Paul Lloyd 

CITY DISPOSAL 

Interviewees 

Marcia Moss, Director Human Resources 
Marvin Hill, General Manager 
Stacey Lipsey, program participant 
John ???, program participant J 
Rafeea Williams, instructor 

1. Education Needs 

The initial impetus of the program here was the need to raise reading 
levels, both to address job literacy problems connected with form-filling 
and for promoting individual educational development. “Literacy was our 
focus”, according to the Human Resources Director, who described a 
range from "non-readers to those uncomfortable reading and writing in 
front of others." The General Manager and the two participants 
mentioned more specific needs related to obtaining the Commercial 
Driver’s License (drivers are the highest-paid workers), computer skills, 
and hazardous materials safety. In addition to these specific skills, the 
instructor saw a need to overcome fears of failure and to build confidence. 
The General Manager mentioned that the company plans to continue 
classes after the grant finishes, because drivers are hard to find and it is 
best to “teach those you know are clean.” 

2. Current classes 

The first CDL class, with just three participants, produced four successful 
CDL written tests — because the instructor took the test with her class. 
This level of involvement appears to be typical of the instructor, because 
the other interviewees’ comments about the classes always referred 
immediately to her role. Comments from participants included: "You can 
read the book, but the classroom brings it out” through discussion and a 
teacher who knows it. The General Manager and the Human Resources 
Director have heard excellent reports from other participants, such as 



“This class is not a waste of my time” and ‘This lady is a very good 
teacher. In addition, both participants had taken to heart much of what 
they had learned about trucks, especially their braking abilities, and said 
that they now avoided tailgating trucks or cutting in front of them when 
driving their own cars. 

3. Classroom facilities 

The classroom is a trailer which the company has brought to the 
worksite. This location on site was seen by all interviewees as a great 
advantage: “There’s no temptation to drive off home instead of to class.” 
However, the small size of the trailer is a problem {“The fellas are kinda 
big.”) and has become more so as demand for classes has increased. The 
General Manager commented: “If I'd known, I'd have gotten a bigger 
trailer.” The solution at present is to have multiple classes— and waiting 
lists. In addition, one of the participants mentioned the inconvenience of 
having no bathroom close to the trailer. The instructor was impressed by 
the response to any requests that she had about locks or electricity: “If 
ever I need anything, they’re right here." 

4. Teaching materials 

Only the instructor and the participants commented on this area. The 
participants had minor criticisms of the CDL computer program, saying 
that some of the pictures were not clear enough on the screen to tell, for 
example, whether a leaf spring was broken or a tire was flat. They also 
suggested that it would be valuable to them to be able to take a copy of 
the program home to work on. And one participant suggested that some 
hands-on tours of a real truck would help those unfamiliar with big rigs. 

The instructor was generally pleased with the Communications course 
materials, commenting that the manual contained all the basic 
information but also allowed the opportunity to expand beyond the 
manual. She had found that the difficulty level was about right for all the 
students (except one low-level reader who needed special individual 
attention). 
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5. Instructor nrlorttlps 

The instructor saw her top priority as planning for ideas to involve the 
students, because she saw communication and confidence-building as 
ver y important. She commented that some students were ashamed of 
their inability to read and write, but she described their speaking and 
listening skills as “outstanding — they surprised themselves.” For the CDL 
class, she emphasized knowing the material (which was new to her), but 
pointed out that “the CDL is reading comprehension”, which related to 
her background in reading education. She summed up her attitude to 
teaching in the words: “Respect yourself and respect your students.” 

That this attitude had come across to the participants was clear from their 
comments, which included: "She is prepared and on time” and “She’s a 
good teacher— with a problem, she’ll take time to help out.” They were 
particularly impressed that she had taken the CDL test with the first class: 
“The lady cares.” The General Manager and the Human Resources Director 
had also heard such comments and emphasized the instructor’s patience, 
non-intimidating personality, and good rapport with her students. 

In addition, the instructor regularly prepares a 20-minute cassette tape to 
guide the work of one student who is virtually a non-reader. This extra 
effort and dedication on the instructor’s part has been noticed by many, 
and was a word-of-mouth recruiting aid. 

6. Recruitment 

The main formal method of recruiting was through newsletters (two 
issues) to employees for several weeks before classes started, with sign-up 
sheets by the time clocks. In addition, an open house with refreshments 
was scheduled to display the new classroom trailer. This “has worked 
fine” according to the Human Resources Director, although only three 
workers signed on for the first class. Their feedback to others has now 
generated more interest and the classes are filling up. A company bonus of 
$300 for completion of a worker’s first class was mentioned by both the 
Human Resources Director and the General Manager as an afterthought, 
but they were doubtful of its influence as an incentive — workers were still 
waiting for the class they wanted rather than just signing on to obtain the 



bonus. This view was confirmed when neither participant mentioned the 
bonus; they referred to the flyer by the time clock, individual approaches 
by other workers, and the open house held in the trailer classroom, 
where workers could meet the instructor and Wayne State personnel as 
part of developing trust in the new system. The General Manager 
estimated that 40-50 employees of the 100+ on the site would go through 
the classes. However, one of the participants mentioned a problem with 
scheduling: “People wanted to do the class, but the timeframe was wrong” 
and suggested the need to survey potential participants about this. 

7. Connection between class and workplace 

The CDL classes had a clear connection with the workplace and worker 
advancement. One participant commented: "The more you can do for the 
company, the more they’ll do for you." With regard to the Communications 
class, the Human Resources Director anticipated an increase in skills at 
filling out forms, route sheets and tracking reports, as well as talking over 
the radio, noting that the class uses company forms. The instructor already 
saw changes: “They are eager to start the newsletter [a class activity] and 
they are more relaxed and willing to speak up." 

In summary, all interviewees thought that the program was going well and 
meeting company and employee needs. The General Manager summed up 
their attitude with: “I’ve seen no negatives." This supportive attitude to the 
program is due in no small measure to the hard work and interpersonal 
skills of the Project ALERT Program Coordinator, who has made herself 
known to many of the workers on the site and is clearly well-liked by all 
parties. The strong rapport and working relationship between the Program 
Coordinator and City Disposal personnel was apparent from the wave of the 
first truck driver we encountered on arrival at the site through meetings 
with managers, the instructor and learners. 



Relation of City Disposal classes to program objectives under Goal 1 

(To design, develop, and implement innovative workplace literacy programs that are tailored to the 
organization, and the skills and cultural backgrounds of participants.] 
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Objective 1.1. To conduct a comprehensive pre-design analysis of jobs, employees, and 

organizations to gather data requisite to the design of effective literacy training. 

All stakeholders are agreed on the nature of employer and employee needs and that these needs 
are being met. 

Objective 1.2. To collaboratively design and develop workplace literacy programs that are 

responsive to the unique needs of the organizations, jobs, and employees. 

Both the CDL and Communications classes are seen as relevant to the workplace and 
appropriate for the workers as individuals. 

Objective 1.3. To design and develop a range of culturally sensitive instructional methods and 
materials that are both technology and non-technology oriented. 

This does not appear to be an issue at City Disposal; no comments were made about it. The 
sensitivity and hard work of the African-American instructor provides an important human 
element here. 

Objective 1.4. To design resource-rich classrooms that support traditional and technology-based 
instruction. 

The trailer provided by City Disposal is adequate to its purpose. The only qualifications to this 
concerned its small size and the lack of a bathroom. 

Objective 1.5. To recruit participants, implement the site-specific programs and provide for 
reinforcement and transfer of training to the workplace. 

Recruitment has been successful, the classes and their instructor have been well received, and 
some transfer to the workplace is evident (with the results of the CDL test from the first class). 
Enrollment in some classes might be increased, possibly by program developers asking 
satisfied students to play a more formal role as one-on-one recruiters. 
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Project ALERT 

Report on Site Visits, March 1996 

Larry Mikulecky and Paul Lloyd 

CHRYSLER DETROIT AXLE 

Interviewees 

Ibrahim Abouarab, UAW Chiysler liaison 
Juanita McLln, UAW Chrysler liaison 
Roberta Walker, President UAW Local 96 1 
Carolyn Peckham, NTC 
Tony Mason, Plant Manager 
Greg Gilett, Production Manager 

1. Education Needs 

The main needs were seen by all interviewees as being communication 
skills and computer skills (to operate machines under Computer 
Numerical Control). The Plant Manager reported that 25-30% of the 
machines in the plant were new, and more were coming — with the total 
rising to about 80% over the next few years. Preparation for employees to 
upgrade skills to operate these machines was seen as key by managers. 
However, the UAW liaisons wanted to see communication skills addressed 
first: “Lack of communication between shifts leads to lots of scrap.” For 
example, if no note or an unclear note was left for a later shift about a 
machine problem, the machine could be run through its process when it 
ought not to be. This problem is partly due to the number of workers 
whose first language is not English— they include Arabs. Albanians and 
Yugoslavs. Also, as the Plant Manager pointed out, apart from 300 new 
hires, the average age of the workforce is in the mid-40s and “a lot of 
them barely made it out of high school.” Their attitude to training has 
been: “I must be broke — they want to fix me." However, the approach taken 
by Project ALERT was intended to alleviate these fears, through meetings 
of all players, and “make the workers comfortable with the idea.” 

2. Delay in starting classes 

Implementation of the first classes had been delayed for some weeks, at 
first because of a threatened strike and then because of a rethink about the 
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Communications class. According to the Plant Manager, "we saw another 
thing — communication, problem-solving, conflict resolution. We’d rather 
they slipped [in timing] than gave us the wrong thing." But the delay had 
caused some embarrassment to the UAW liaisons; they were being asked: 
“What happened to the classes?" The President of the UAW local thought 
there had been unnecessary delays in visits to the plant floor, but felt that 
overall there had been good, frequent contact with the Wayne State team. 

3. Classroom facilities 

The Production Manager had high praise for the help that the project 
had given them in designing the new classrooms, “which we use a lot.” 
One contained 12 computer stations and the others were fitted up well 
for class or small group teaching. 

4. Teaching materials 

Both the Plant Manager and the Production Manager were “very 
impressed so far” with the interactive multimedia they had seen. They 
saw it as being very helpful, and possibly central, for CNC training and for 
other set-up processes. If a process could be captured on multimedia, 
that would be a way “to get someone comfortable on a new job” without 
making expensive mistakes. However, the Plant Manager saw a possible 
problem with proprietary information if the project wanted to use 
elsewhere materials developed for Chrysler. The Production Manager, 
however, believed that “most information is pretty generic" and that this 
may not be a difficulty. Both of them emphasized that the 
Communications class was just a means toward the end of the pre-CNC 
class and the “multimedia CNC simulation.” Maintaining timelines with 
the multi-media part of the curriculum development appeared very 
important to the managers. 

5. Instructor priorities 

Not applicable at this stage, because classes are not yet in operation. 

6. Recruitment 

A comprehensive recruitment plan has been developed and will be 
implemented mainly by the Plant Liaisons. They will approach workers in 
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order of seniority, because, as one of the liaisons put it, “the higher group 
has been ignored a long time." The President of the UAW local also 
pointed out that this group is more stable — there will be “no worry about 
lay-offs or shifts to other jobs." Fifty workers will be targeted initially. 
Preliminary work has begun on this, including the production of a flyer, 
and many workers are ready for the program to start. As noted earlier, 
the liaisons are being asked: “What happened to the classes?” and the 
Production Manager added that “we seem to have hit a nerve out there.” 

7. Connection between class and workplace 

Not applicable at this stage, because classes are not yet in operation. 
However, content promised by multimedia curriculum appears to be 
strongly connected. 

In summary, all interviewees thought that preparation for the program was 
going well, apart from the delays mentioned above. And there was a real 
excitement about the program’s possibilities, particularly the multimedia 
package for the pre-CNC class. But further delays could present major 
problems. 

Relatio n of Chrysler Detroit Axle classes to program objectives under Goal 1 

[To design, develop, and implement innovative workplace literacy programs that are tailored to the 
organization, and the skills and cultural backgrounds of participants.] 

Objective 1.1. To conduct a comprehensive pre-design analysis of jobs, employees, and 

organizations to gather data requisite to the design of effective literacy training. 

All stakeholders are agreed on the nature of employer and employee needs and that these needs 
will be met. The emphasis differed somewhat between managers and union representatives. 

Obiective 1.2. To collaboratively design and develop workplace literacy programs that are 

responsive to the unique needs of the organizations, jobs, and employees. 

Both the Communications and pre-CNC classes are seen as relevant to the workplace and 
appropriate for the workers as individuals. Delays in development are currently being accepted, 
but are starting to become a problem. 
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Objective 1.3 . To design and develop a range of culturally sensitive instructional methods and 
materials that are both technology and non-technology oriented. 

This issue has been addressed by choosing suitable UAW plant liaisons, including one from 
the large Arab minority at the plant. Also a short section in the Communications Skills 
instructor’s manual gives advice on ways to establish a learning environment in which diversity 
is valued. 

Objective 1.4 . To design resource-rich classrooms that support traditional and technology-based 
instruction. 

The multimedia and other classrooms provided by Chrysler Detroit Axle are of high quality and 
very suitable for their purpose. 

Objective 1.5 . To recruit participants, implement the site-specific programs and provide for 
reinforcement and transfer of training to the workplace. 

Not applicable at this stage, because classes are not yet in operation. A strong plan and a 
recruiting brochure are in place. 
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Project ALERT 
Outline of Evaluation Plan 



Larry Mikulecky and Paul Lloyd 



Evaluation goals: 

1 . To assess achievement of project goals and objectives. 

2. To assess impact of program: learner gains, work productivity. 

3. To conduct a year-by-year cost/benefit analysis of the project. 

1. To assess achievement of project goals and objectives. 

a. Review objectives and deliverables. 

b. Collect documents and information that support achievement of project goals and objectives 

(see list following). 

2. To assess impact of program: learner gains, work productivity. 

a. Assist with development of appropriate measures. 

b. Monitor gathering of assessment data on learners by project personnel (including pre/post 



3. To conduct a vear-bv-vear cost/benefit analysis of the project. 

a. List deliverables and other benefits: 

e.g., classes held, 

number of learners served, 
course materials and instructor guides, 
computer programs developed, 
guidelines for others to implement model. 

b. For all project staff, list 

the benefits anticipated, 

when they are expected to be achieved, 

what progress has been made, and 

how much time and resources have been spent in relation to deliverables. 

c. Divide deliverables into Service Provision (e.g., number of learners served), and Replicable 

Model (applicability of the teaching model). 



measures). 
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Project ALERT 

Sources of Information to Assess Achievement 
of Program Goals and Objectives 

General 

Proposals (original grant proposal, continuation proposals, human subjects review, resumes for 
project personnel) 

Analysis of costs and benefits (project tracking sheets, report on allocation of team members’ time, 
costs of staff time and expenses, yearly summary of achievements for each goal and objective) 

[Goal 1. To design, develop, and implement innovative workplace literacy programs that are tailored to the 
organization, and the skills and cultural backgrounds of participants ] 

Objective 1,1 . To conduct a comprehensive pre-design analysis of jobs, employees, and 

organizations to gather data requisite to the design of effective literacy training. 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 
Evaluator interviews with project personnel and participants 
Evaluator visits to program sites 

General print information from workplace sites (e.g., Davis Tool description, Detroit Axle 
description, UAW-Chrysler Course Summary) 

Needs assessment process (flow chart, data collection matrix, task analyses, job descriptions) 

Instruments (hourly employee interview, supervisor interview, observation checklist) 

Needs assessment reports (data-base output from each site) 



Objective 1,2 . To collaboratively design and develop workplace literacy programs that are 

responsive to the unique needs of the organizations, jobs, and employees. 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 
Evaluator interviews with project personnel and participants 
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Evaluator visits to program sites 

Whole language curriculum materials (curriculum development task list, job task analyses, course 
description, lesson plans, workplace resource materials, supplemental materials, instructor’s 
manual) 

Technology curriculum materials (curriculum development task list, overview plan, instructional 
strategy, completed software evaluation forms, list of software purchased, job task analyses, 
in-house software developed) 



Objective 1.3 . To design and develop a range of culturally sensitive instructional methods and 
materials that are both technology and non-technology oriented. 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 

Evaluator interviews with project personnel and participants 

Evaluator visits to program sites 

Reviewing curriculum for bias and accounting for learning styles and workplace culture (reports on 
process and results: e.g., article on methods with examples) 

Whole language curriculum materials (curriculum development task list, job task analyses, course 
description, lesson plans, workplace resource materials, supplemental materials, instructor’s 
manual) 

Technology curriculum materials (curriculum development task list, overview plan, instructional 
strategy, completed software evaluation forms, list of software purchased, job task analyses, 
in-house software developed) 



Objective 1.4. To design resource-rich classrooms that support traditional and technology-based 
instruction. 

Site plans (including relation to needs assessment) 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 
Evaluator interviews with project personnel and participants 
Evaluator visits to program sites 



Objective 1.5 . To recruit participants, implement the site-specific programs and provide for 
reinforcement and transfer of training to the workplace. 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 
Evaluator interviews with project personnel and participants 
Evaluator visits to program sites 
Recruitment materials (plans, flyers) 

Site plans, classes held, learners served (including relation to needs assessment) 

Instructor recruitment materials 

Delivery documentation (database reports, instructor reflections and logs) 

Instruction follow-up (review session materials, job aids) 



Goal 2 To conduct project evaluation and program impact research. 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 

Project data base (description, output) 

Needs assessment reports (data-base output from each site) 

Assessment (instruments, pre/post test results, effect on job performance) 

— student course evaluations ? 

Impact research model (plan, matrix, flow chart, task list) 

Impact research results (analysis of instructor logs, reports on sites and learners) 

Evaluator interviews with project personnel and participants 

Evaluator visits to program sites 

External evaluation (program impact measures, reports on achievement of project goals and 
objectives, cost/benefit analysis) 
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Goal 3 . To disseminate the program, program products, and research findings. 
Project meeting minutes 

Conference papers (proposals, summaries) — these should be submitted to ERIC 
and for possible publication 

Documentation of replicable model (procedures, curriculum) — are there any others? 



Goal 4 . To promote program institutionalization and diffusion. 

Project meeting minutes 

Notes from project team site visits (include estimates of time spent on various activities) 
Documentation of replicable model (procedures, curriculum) — are there any others? 






Full List of Information Sources for Evaluation 

Proposals (original grant proposal, continuation proposals, human subjects review, r£sum£s for 
project personnel) 

Analysis of costs and benefits (project tracking sheets, report on allocation of team members’ time, 
costs of staff time and expenses, yearly summary of achievements for each goal and objective) 

Project meeting minutes 

Notes from project team site visits (including estimates of time spent on various activities) 

General print information from workplace sites (e.g., Davis Tool description, Detroit Axle 
description, UAW-Chrysler Course Summary) 

Needs assessment process (flow chart, data collection matrix, task analyses, job descriptions) 

Instruments (hourly employee interview, supervisor interview, observation checklist) 

Needs assessment reports (data-base output from each site) 

Whole language curriculum materials (curriculum development task list, job task analyses, course 
description, lesson plans, workplace resource materials, supplemental materials, instructor’s 
manual) 
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Technology curriculum materials (curriculum development task list, overview plan, instructional 
strategy, completed software evaluation forms, list of software purchased, job task analyses, 
in-house software developed) 

Reviewing curriculum for bias and accounting for learning styles and workplace culture (reports on 
process and results: e.g., article on methods with examples) 

Recruitment materials (plans, flyers) 

Site plans, classes held, learners served (including relation to needs assessment) 

Instructor recruitment materials 

Delivery documentation (database reports, instructor reflections and logs) 

Instruction follow-up (review session materials, job aids) 

Project data base (description, output) 

Assessment (instruments, pre/post test results, effect on job performance) 

— student course evaluations ? 

Impact research model (plan, matrix, flow chart, task list) 

Impact research results (analysis of instructor logs, reports on sites and learners) 

Conference papers (proposals, summaries) — these should be submitted to ERIC 
and for possible publication 

Evaluator interviews with project personnel and participants 
Evaluator visits to program sites 

External evaluation (program impact measures, reports on achievement of project goals and 
objectives, cost/benefit analysis) 

Documentation of replicable model (procedures, curriculum) — are there any others? 
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Wayne State Project ALERT 



FORMATIVE EVALUATION REPORT 

AUGUST 1995 



Larry Mikulecky and Paul Lloyd 
Indiana University 



Introduction 

This formative evaluation report is based on information gathered 
mainly during July and early August 1995. It should be regarded as a 
snapshot of Project ALERT taken at this time. The project has since 
made progress not recorded here on a number of the issues 
mentioned in the report. 

Funded in late 1994 by the National Workplace Literacy Program 
(NWLP), Project ALERT is a partnership of Wayne State University’s 
College of Education and Labor Studies Center, the Detroit Public 
Schools, and the unions and management at several Detroit 
manufacturing firms. These are the Chrysler Detroit Axle Plant and 
UAW Local 961, the UAW-Chrysler National Training Center, the Davis 
Tool & Engineering Company and UAW Local 174, and the City 
Management Corporation. 

According to the grant proposal, the stated purpose of the project 
is: 



“To provide for an innovative institutionalized workplace literacy 
program that will increase the skills of employees and improve 
the productivity of the organization and the quality of its 
products” (p. 16). 

The proposal further breaks the purpose out into goals and objectives 
as follows (pp. 16-17): 
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Goal 1: To design, develop, and implement innovative workplace 
literacy programs that are tailored to the organization, and the 
skills and cultural backgrounds of participants. 

Objective 1: To conduct a comprehensive pre-design analysis of 
jobs, employees, and organizations to gather data requisite to 
the design of effective literacy training. 

Objective 2: To collaboratively design and develop workplace 
literacy programs that are responsive to the unique needs of 
the organizations, jobs, and employees. 

Objective 3: To design and develop a range of culturally sensitive 
instructional methods and materials that are both technology 
and non-technology oriented. 

Objective 4: To recruit participants, implement the site-specific 
programs, and provide for reinforcement and transfer of 
training to the workplace. 

Goal 2: To conduct project evaluation and program impact research. 

Goal 3: To disseminate the program, program products, and 
research findings. 

Goal 4: To promote program institutionalization and diffusion. 

The project was awarded the federal grant to carry out this 
proposal in November 1994, which was two months earlier than the 
project team had expected. A number of project members had prior 
commitments to finish with before starting work on this project and 
there were also some necessary changes and additions to the team. 
Though some work on portions of the project occurred throughout the 
winter, a full team with clear leadership and team responsibilities was 
not in place until March 1995. 

One casualty of changing leadership and shifting team member 
responsibilities was the project team’s communication with the 
external evaluators, which was infrequent for several months. 

Meetings and faxes in late 1994 and early 1995 produced a To Do List 
for the project team, which asked for: 

• a breakout on costs to goals, 

• a list of project deliverables divided into Service 
Provision and Replicable Model, and 
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• notes of meetings of project personnel. 

“Service Provision” was seen as a convenient way to consolidate 
program delivery and evaluation goals and “Replicable Model" a 
convenient way to consolidate the dissemination, institutionalization, 
and diffusion goals. These requests were not addressed by the project 
team until late spring of 1995. 

During May 1995, contact was re-established and a more regular 
flow of documents began, including minutes of project meetings. The 
evaluators could then see that the project team had been completed, 
that there had been much progress on liaison with the workplace 
partners, and that the whole language curriculum was in an advanced 
state of development preparatory to teaching the first class at Davis 
Tool. However, it was clear that the time originally proposed for 
starting this class would not be achieved; the delay was likely to be 
about three months. 

In July 1995, the evaluators visited Detroit to meet with the project 
team. During detailed discussion of the progress made to date, the 
evaluators appreciated the reasons why the schedule had slipped, but 
urged the team to work toward some concrete deliverables for Year 1 , 
including documentation, curriculum tools, and lessons learned for 
producing a replicable model. The evaluators emphasized this because 
the service provision portion of achievements will be at most one 
workplace class taught before the end of October. The project team 
was encouraged to document the methods and techniques used in 
workplace liaison, needs assessment, curriculum development, 
teacher recruitment, instructional delivery, and learner assessment. 

An idea suggested for deliverables related to project goals 3 and 4 
(Replicable Model) is the production of draft chapters of a guide to 
setting up such a project. These deliverables could demonstrate the 
Year 1 progress in developing a model for an innovative workplace 
literacy program which could be institutionalized, replicated, and 
disseminated. 

Since the July meeting, there has been further contact between the 
project team and the evaluators, including discussion of draft 
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curriculum and assessments for the first class at Davis Tool, and the 
conducting of extended formative evaluation interviews with key 
project personnel. 

Evaluation Goals and Sources of Information 

The purpose of a formative evaluation is to monitor the progress of 
a program while it is still under way, in order to recommend changes 
needed to help the project team keep the program on track. Four 
areas often examined in such evaluations are program goals, resources, 
processes and impact. By early August, no classes had been held and 
curricular materials were still being developed. For these reasons, the 
evaluation will concentrate primarily upon consensus of goals and 
sufficiency of resources. 

It is an assumption of this evaluation model that, for a program to 
be most effective, program goals should be shared and understood by 
all stakeholders, including such groups as the principal investigators, 
curriculum developers, company personnel, and union officials. The 
rationale for this assumption is that, when goals are shared and 
understood by all, communication problems are reduced, teams can 
work more efficiently, and the likelihood of goals being reached 
increases. Sometimes, misunderstandings may occur as to what a 
program is supposed to accomplish. These disagreements need to be 
clarified and resolved before the program continues very far. 

In addition, formative evaluation should address a program's 
resources, such as time, work space, materials, and expertise. Are 
sufficient resources available to meet the stated program goals, and is 
the program making reasonable progress toward these goals? 
Resources can include time for curriculum development, class time 
allocated to achieve goals, appropriate learning materials and space for 
instruction, and personnel with the training and expertise needed to 
accomplish goals. 

The evaluators have gathered evidence of the project's activities 
and progress through documents, observations and interviews. The 
project team has sent evaluators a number of relevant documents, 




4 183 



including the original project proposal, minutes of team meetings, 
instruments used for the needs assessment, and samples of the whole 
language curriculum and its learner assessments. The evaluators 
learned more about the project on a trip to Detroit to talk with the 
team and visit some of the workplaces involved in the project. And. 
Anally, the evaluators conducted extended telephone interviews, in 
which the project personnel were asked about program goals and 
resources. The following project personnel participated in individual 
interviews: 

Dr Dale Brandenburg, Co-Principal Investigator. 

Dr Gary Powell, Co-Principal Investigator. 

Dr Rita Richey, Co-Principal Investigator. 

Dr Hal Stack, Co-Principal Investigator. 

Ms Nancy Ruetz, Assistant Project Director of Curriculum. 

Ms Mary Jarvis. Graduate Research Assistant (Reading). 

Ms Irene Sinclair, Program Coordinator, 

Ms Sheila Reed, Project Secretary, 

Mr Steve Nelson, Davis Tool Human Resources, and 
Mr Mike Wysocki, Davis Tool Union Chairman. 

Interviews consisted of speciAc questions followed by standard 
non-directive prompts (i.e., Can you think of any more examples?) as a 
means for prompting recollection and full answers. The speciAc 
questions that the interviewees were asked are as follows: 

Goals 

1. What do you see as the main purposes, goals and objectives of 
the project? 

2. Can you give me some examples of what you see the project 
producing? 

Resources 

For a project to be fully effective, it needs sufAcient resources. 

This means people, time, space and materials. Can you comment 

on how you see the resources of this project, giving speciAc 

examples? 

1. People (background, training, etc) 

2. Time to reach goals 

3. Work space (development, delivery) 

4. Materials (development, teaching) 
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assessment. The information in this needs assessment is gathered in 
such a way that all key stakeholders are contacted and results are of 
use for both curriculum development and later impact research. This 
flow chart is an outgrowth of the experience of personnel in 
contacting business and union partners. In addition to documenting 
design and development accomplishments, the flowchart will be of use 
in project goals related to producing a replicable model. Project 
personnel have also developed several interview forms for use at the 
workplaces: an hourly employee interview, a supervisor interview, and 
an observation checklist. The evaluators have seen blank copies only, 
although the forms have been used at Davis Tool. Prototype results 
from the data-base output include samples of the information gathered 
at Davis. 

For the whole language curriculum, a complete set of lesson plans 
for the Test-Taking class has been reviewed, plus about 50% of the 
projected lessons for the Reading and Math classes, as well as 
instruments for learner assessment of the class on test-taking. The 
lessons are clearly designed, are linked to learning goals established 
with business partners, and provide enough detail to be of use to 
teachers. The lesson framework is also replicable and should prove 
useful in the continued development of lessons. 

For the interactive technology curriculum, the only documents seen 
so far are an overview of the curriculum plan and some sample 
software evaluation forms. The overview describes, in general terms, 
what the custom-designed job simulations will look like and how they 
will be supplemented with commercially available software. The 
overview also lists the desirable characteristics of the technology 
classrooms. The software evaluation form contains rating scales for 
assessing the value of commercial software to the project. The major 
areas rated are learner/ computer interactions, learner control, and 
sequencing of instructional events. This general material is 
professional and appears to be useful. However, a significant amount of 
project resources have been allocated to the technology curriculum. 
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The evaluators are concerned about a mismatch between allocated 
resources and progress toward deliverables in this area. 

Observations and Conversations: During the visit on July 19, the 
evaluators discussed with the project team a full range of issues 
connected with program design and development, and also made site 
visits to Davis Tool and the UAW-Chrysler Wayne Center. Information 
gathered through these observations and conversations presents a 
picture of a good deal of effort on the part of project personnel — but 
also a good deal of frustration. Decisions with businesses were often 
slow in coming, contact people at sites changed jobs, and on more 
than one occasion efforts at reaching agreement which took many 
meetings and dozens of hours were lost due to budgetary and 
administrative priority changes. Though these setbacks are 
discouraging in relation to service provision goals, careful 
documentation of starts, false-starts, and dead stops can be of use in 
documenting for others the stages and frustrations in developing 
cooperative workplace literacy programs. 

Project personnel reported that the needs assessment had been 
completed at Davis Tool and that, using the flow chart mentioned 
above, needs assessments were now being conducted at Chrysler 
Detroit Axle, City Management, and the Wayne Center. 

Regarding curriculum, the project team reported that the test- 
taking module of the whole language curriculum at Davis Tool was 
complete, and that the other two modules — math and reading — were 
making good progress. For the interactive technology curriculum, it 
was reported that the high-tech classrooms had been installed at 
Detroit Axle and the Wayne Center, and that off-the-shelf literacy 
software was being reviewed as supplementary instructional material, 
to use until the project’s own custom-designed software is ready. 

The evaluators’ site visit to Davis Tool revealed management and 
union partners who were enthusiastic about the project, and were very 
cooperative with the curriculum developers. The proposed teaching 
area, two rooms up a stairway at one end of the factory floor, was still 



in preparation. The larger room contained tables, chairs and a 
chalkboard, but was not yet fully set up as a classroom. The smaller 
room, which is to be used for individual consultations with workers, 
was still in use as a storage area, but has since been cleared and made 
ready. 

A visit was also made to the Chrysler Wayne Center. The Wayne 
Center offers a wide range of classes and other educational 
opportunities to Chrysler employees, family, and the local community. 
In July, negotiations were under way to determine how Project Alert 
and the Wayne Center might best work cooperatively. It was clear that 
meetings and discussions had transpired and evaluators were shown 
rooms and computer facilities which might be made available once 
learner populations and goals had been established. Several possible 
populations and classes were mentioned. Later, in August, the 
evaluators were notified that, due to budget cuts, this location would 
be included in the service provision portion of the project through the 
second year only, and dropped in the third year. 

Other topics of discussion with project personnel related to plans 
and concerns for curriculum development and delivery as well as 
recruitment of learners and of classroom teachers. It was clear that 
the team was aware of what needed to be done and was involved in 
solving problems to accomplish project goals. 



To reflect the ethnic diversity at the participating workplaces, 
Arabic and African American union representatives were selected as 
liaisons specifically for this project at Detroit Axle, and a recruiter was 
identified at Davis Tool. Regarding instructor recruitment, it was 
reported that the project team was working hard to find a teacher for 
the whole language curriculum at Davis Tool, but was having some 
difficulty finding a suitable person. By the second week of August, an 
instructor had been recruited, and a pool of six to eight other possible 
candidates had been found. 

Formal Interviews: In their individual interviews, project personnel 
were in general agreement on the importance of curriculum 
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development and cooperating with the workplace partners to enhance 
workers’ skills. Interviewee’s statements indicating this recognition of 
curriculum development and service delivery as an important project 
goal included: 

“The initial emphasis is on curriculum development.” 

“One goal is to deliver models of instruction based on 
separate philosophies and media, accommodating the 
workplace sites.” 

“To design resource-rich classrooms and establish bonds 
with our workplace partners.” 

“We aim to deliver good services to real people to make a 
difference.” 

“To improve the basic literacy skills of the entire workforce, 
and make them better at their jobs.” 

‘To bring an instructional program that meets the needs of 
employers, employees, and the promises of the grant.” 

Goal 2: Evaluation and Impact research 

Documents: During the July meeting, the project team shared with 
evaluators a conceptual framework for conducting impact research. 

The framework broadly considered impact in terms of both learner 
gain and organizational impact, including a comparison of the project’s 
alternative delivery systems. At the time of the visit, instruments had 
not yet been fully developed. Since the July visit, the project team has 
developed for the class at Davis Tool a pre/post evaluation plan which 
includes a mix of standardized measures (i.e., the TABE Math and 
Reading), custom-designed measures matched to the class curriculum, 
and a series of questions and rating scales to assess attitudinal 
changes. The Davis Tool class is being designed to address the need 
for workers to pass a company selected test being used as an indicator 
for access to particular jobs. During telephone conversations in July, 
project personnel discussed the possibility of setting up a control 
group of employees not able to enter the first class. This group could 
also be pre- and post-tested and compared to the class group on gains 
as well as performance on the company-selected test. It was later 
learned that this suggestion was vetoed by personnel at Davis Tool. 
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The mix of multiple measures selected and developed by the 
project team is appropriate to the class and to the larger conceptual 
framework shared during the July meeting. Solid dates for the class 
beginning and duration have not yet been communicated to the 
evaluators. We have some concern about whether there will be 
sufficient time for gathering pre- and post-data on what is likely to be 
the only class delivered by the project during its first year. 

In addition to pre- and post-data, project personnel are developing 
a data-base for compiling learner and task assessment data. A mock-up 
of data-base output has been shared with the evaluators. Though it is 
too soon to tell conclusively, the data-base shows promise as a tool for 
being able to consolidate and manipulate assessment information for 
the large number of learners involved in the project across multiple 
sites. The data-base should also serve as a new tool available to other 
programs attempting to replicate the Project Alert model. 

Observations and Conversations: At the July 19 meeting, the 
project team was mainly focused upon the immediate concerns of 
curriculum development, hiring instructors, and securing from work- 
site partners final understandings about classes. With the exception of 
the general conceptual model for conducting impact research, the 
evaluators were able to observe little in terms of an impact evaluation 
infrastructure. Concern about this was expressed during the visit. 

Since that time, a good deal more work appears to have been done. 

Formal Interviews: Only a few of the project personnel interviewed 
mentioned measuring the impact of the program as a major project 
goal. These comments come mainly from the Co- Principal 
Investigators. All the comments on this topic are given here: 

‘To test the output of curricula in a variety of workplace 
settings: is it effective?" 

“To compare instructional approaches.” 

“To determine impact.” 

“Testing to see what works." 
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It can be argued that evaluation is mainly the concern of the Co- 
Principal Investigators and need not concern curriculum developers, 
instructors, supervisors, and others. And it may be that other project 
personnel consider evaluation a goal, even though they did not 
mentioned it after repeated probes of, “Can you give me any more 
examples?” 

Goals 3 and 4 : Creating and Disseminating a Replicable Model 

Goal 3: Dissemination 

Documents: Little has occurred in this area as yet, but a first 
conference presentation arising from the project has been delivered. 
Gary Powell and Lauri Elliott presented at the Adult Literacy and 
Technology Conference held in Philadelphia during August. The 
evaluators were given a copy of their proposal. 

Observations and Conversations: In discussions during the July visit, 
the value of documenting project methods and techniques as they 
occur was emphasized by the evaluators. This will serve both to 
capture the processes while they are fresh and to show progress on 
deliverables connected with the replicable model. Plans for carrying 
out this documentation were discussed at the project’s meeting on 
July 28. 

Formal Interviews: None of those interviewed mentioned as a goal 
documenting the program model in ways that will allow it to be 
disseminated. However, several did mention this area in connection 
with “Examples of products and deliverables” below. 

Goal 4: Institutionalization and diffusion 

Documents and Observations: The evaluators have seen little 
evidence in this area, but the project team reports that the curriculum 
developers have been keeping logs of their activities, which they have 
used in team discussions about institutionalizing the model. 

Formal Interviews: Only a few of those interviewed mentioned 
documenting the program model in ways that will allow it to be 
institutionalized. All the comments on this topic are given here: 
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“In the process of institutionalization, the organizations 
[workplaces] need to absorb the project’s aims." 

"To produce a replicable model.” 

"To introduce and institutionalize, extracting from 
experiences — particularly in joint labor/ management 
cooperation, and to diffuse more broadly. D.P.S. has a role in 
continuing after this project.” 

Examples of products and deliverables 

Documents: Considering the two main types of deliverable — 
curriculum materials and a documented model, evidence gathered by 
the evaluators so far relates mainly to curriculum development, 
including lesson plans and learner assessment for the first class at 
Davis Tool. These appear to be of high quality, and very relevant to the 
needs and objectives of the workers at Davis. For the program model, 
materials seen to this point include organization charts and needs 
assessment tools. In order for other programs to use these assessment 
tools, cover sheets with instructions for interviewing and use with 
different populations would be helpful. 

Formal Interviews: Again there was general agreement among team 
members on what products they expected to see as a result of the 
project: most interviewees mentioned curriculum materials and a 
documented model. However, in reference to the model, the aspects 
mentioned were very diverse, usually reflecting the particular role that 
the interviewee played in the project. For example: 

“Journals and stories of development at the workplace, 
model for needs assessment.” 

“Knowledge and information about the learning process: 
transfer, improved literacy skills, productivity.” 

“Assessment tools and instruments, ‘how to’ advice on union 
liaison, ongoing capacity in D.P.S. to do new things, people 
with skills that make a difference.” 

“Instructor recruitment strategies.” 

“Assessment tools — pre/post, scenario on SPC; log/guide on 
development." 
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“Environment to continue training at workplaces beyond 
grant, staff development." 

“Workable model taking account of workplace realities, 
delivering high quality training." 

A few interviewees mentioned additional products of a less tangible 
nature. These included producing workers with improved skills and 
developing environments at the participating workplaces conducive to 
later institutionalization of the program model. 

Resources: personnel (background, training, etcl 

Documents: The evaluators have received copies of the resumes of 
all four Co-Principal Investigators, and these show that the nature and 
range of expertise of these members of the project team are suited to 
this project. They include a mix of instructional and workplace 
experience which should blend well together in carrying out the 
project’s goals. Dale Brandenburg has considerable expertise in 
training evaluation and economic development, Hal Stack brings a 
wealth of experience in labor-management relations and the setting up 
of workplace programs, Gary Powell contributes his expertise in 
relation to multimedia and instructional design, and Rita Richey has 
considerable experience in needs assessment and impact research. 

Resumes have also been received for Nancy Ruetz and Chauncey 
Cooper. These show that Nancy has much experience with workplace 
programs, both as instructor and administrator, and that Chauncey has 
worked for many years with adult students and workplace classes. 

Observations and Conversations: It appears that those involved in 
curriculum development have a complementary mix of academic 
knowledge and workplace experience suitable for the nature of the 
project. Nancy Ruetz and Irene Sinclair have considerable experience 
in adult education, workplace curriculum development, and basic 
skills instruction. Each has expended considerable effort since joining 
the project to master new information in the project areas. Mary 
Jarvis and Lauri Elliott bring expertise and a knowledge of curriculum 
development, the one in whole language, the other in technology. 
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Formal Interviews: Program personnel are very much agreed that 
they have a strong team together now, with a good mix of expertise. 
This includes the areas of program administration, research methods, 
curriculum development, and workplace liaison. As some of the 
interviewees put it: 

“The team is now sufficient and complete." 

“We have purposely put together a team with different 
backgrounds and expertise.” 

“Generally we have a very good team now with the skills to 
make the project work and to make good use of resources.” 

“We can get expert advice on almost any topic.” 

The only area where recruiting suitable personnel has proved 
difficult is that of worksite instructors. Until early August, it was not 
possible to find a teacher to start the first class at Davis Tool, despite 
extensive searching. (As of August 11, an instructor has been signed on 
for this class and a pool of six to eight other possible candidates 
found.) 

On the other hand, one innovative feature of the project is the 
selection and training of workplace liaisons at the Chrysler Detroit 
Axle plant. These union representatives have been chosen to assist 
with recruitment of workers to participate in the program, and should 
provide a valuable link between the project team and Detroit Axle. 



Resources: time to reach goals 

Documents: Allowing for the start-up date of November rather than 
January, the project is now in its tenth month. Therefore, according 
to the timeline in the original proposal, project activities have fallen 
behind by about three or four months. But every effort is now being 
made to catch up on this delay. 

Observations and Conversations: In addition to the overall delay 
during this first year, the project has a particular problem with the 
technology curriculum, which has not begun development yet because 
liaison with the workplaces where it is to be used is still incomplete. 
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The evaluators discussed this with the project team on July 19, and it 
was agreed that the hundreds of hours required for developing even a 
single hour of computer-based instruction would mean either cutting 
back on the amount of new multi-media curriculum to be developed by 
the project or a major reallocation of resources. Even though this 
portion of the project was delayed, personnel salaries related to this 
goal were not reallocated to other goals or delayed until this personnel 
time could be more productively used. 

Formal Interviews: As described in the introduction to this report, 
the project made a slow start in this its first year. A number of those 
interviewed described the difficulties of the early months, with 
enforced changes in personnel and delays in recruiting a full team. It 
is, however, generally agreed now that the project is in good shape 
and getting back on track. Comments included: 

“At first we were playing catch-up) — we made a slow start, our 
act was not together. But now we are up and running.” 

“We have been scrambling since Day One." 

“It was frustrating with the slow start — we missed the first 
goal of a class by June. But it should go better from now on.” 
“Ouch! We’re not on track. By the end of the third year, we 
will catch up.” 

The one concern specifically mentioned was the technology- based 
curriculum, which has not progressed far enough yet for the project 
team to have a clear idea how that will look eventually: 

“We have a problem in the technology area — time will 
probably become a constraint.” 



Resources: work space for development and delivery 

Observations and Conversations: Of the work space devoted to the 
project, the evaluators saw during their visit only the College of 
Education basement “war room”, the classrooms at Davis Tool, and 
the Wayne Center. The computer lab at the Wayne Center appeared to 
be an excellent facility, well equipped for conducting computer-based 
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learning. The former SPC room at Davis was still in process of 
conversion, but clearly had the potential to become a good learning 
space. 

The curriculum development area in the College of Education, 
known to its inhabitants as the “war room", had a most exciting and 
business-like air, with its pin-up curriculum ideas and whole language 
curriculum team working well together. A potential problem with this 
area was the limited communication facilities: a single telephone line 
had to serve all who worked in the room and prevented their having 
electronic mail or a fax machine. It was mentioned during the visit 
that the project team has begun action to provide extra lines, but 
without positive results as yet. These restrictions appeared to hamper 
the curriculum development team, although they were clearly making 
the best of the situation. 

Formal Interviews: The teaching space available for the project, 
both for the project team and at the workplaces, is agreed to be 
adequate or, in some cases, very good. The teaching rooms with 
computers at the Chrysler Detroit Axle plant appear to be excellent. 
Interview comments included: 

“Work space on site is in good shape — the plants are really 
good at accommodating the project." 

“There is a wide variety of instructional space, but some is 
not settled yet." 

Interview comments also, however, underscored the concern about 
space in the “war room" which was observed during the July visit. In 
addition to communications bottlenecks caused by a single phone line, 
if more project personnel will be working in the “war room" when the 
technology-based curriculum is being developed, the facilities will 
become very crowded. Comments on this issue came from both those 
who worked in this area and those who did not: 

“There are some limitations in the basement, where a 
number of key staff work.” 

“We have two small rooms in the basement and share one 
’phone. It’s very frustrating and tight, and we need an extra 
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line for e-mail. And soon there will be two or three others in 
there, working on the technology curriculum.” 

"We have plenty of space — I have my own office. But the 
basement is cramped — there is only one ’phone line, two 
computers, and the fax is way upstairs." 

Resources: materials for development and delivery 

Formal Interviews: The project seems to have sufficient resources 
for development materials, and team members agree that this has 
presented no difficulties. The arrangements at the worksites sire still 
in progress, but the picture there seems to be good as far as it is 
known. Representative comments about materials follow below: 

"Too soon to tell — first year or so is fine. In Year 2, we’ll need 
some reshaping of the materials budget.” 

“There’s plenty of money in the budget for that — perfect. 

Plenty of software and hardware.” 

“No problem — we have received anything requested." 

Once classes are operating and demand for materials increases, this 
area should be re-examined. 

Conclusion and Suggestions 

The current team for Project ALERT works well together, and is 
generally agreed on what the project is trying to achieve and how well 
the resources available help them toward their goals. This is especially 
true in the area of service delivery and to a lesser extent in terms of 
evaluation and developing a replicable model. Several factors 
mentioned earlier have caused the project to fall behind its projected 
time line. Exemplary progress during the last few months suggests the 
new team has the project moving ahead and may be able to make up a 
good deal of the time lost during the first six months of Year One. The 
overall picture of the project’s progress is positive at this time, but 
there are a number of areas of concern to project personnel and to the 
evaluators. Evaluator perceptions of project strengths and weaknesses 
will be summarized here goal by goal (Goals 1-4, as listed in the 
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original proposal, pp. 16-17), and suggestions made for changes the 
project team should consider. 

Goal Is To design, develop, and implement innovative workplace 
literacy programs that are tailored to the organization, and the 
skills and cultural backgrounds of participants. 

After a slow start, the project team has made great progress in 
working with its first workplace, Davis Tool. At Davis, the needs 
assessment has been completed, development of the whole language 
curriculum is well advanced, and arrangements are in process for 
establishing a suitable teaching area. Recruitment for the first class is 
now under way, and interest is apparently very high among the 
workforce. 

At the other sites, the computer-based teaching areas have been 
established and workplace liaison and needs assessment have begun, 
and appear to be proceeding smoothly. Once the needs of these 
workplaces are known, development of the customized technology 
curriculum will be able to commence. The project team is justifiably 
concerned about this aspect of the project, considering the hundreds 
of hours of development time required for it, and will have a clearer 
idea of how this will work out once the first modules are being 
developed. 

More generally, the possible products from the project all sound 
attractive. It does not seem likely, however, that the project will have 
the resources to produce all of them. The diversity of products 
mentioned suggests decisions have not yet been made about which 
products have highest priority and who has responsibility for 
producing them. If this is the case, such decisions should be made in 
the relatively near future so that project personnel can be gathering 
the information they need to produce such products. 

Materials and work space for the project, both at the worksites and 
for project personnel, are generally satisfactory and sometimes very 
good. Availability of materials is certainly not a problem for the project 
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team, and the participating workplaces have been very forthcoming in 
providing job materials to assist with developing custom-designed 
curriculum. Teaching areas at the worksites should be ready in good 
time for the start of classes, and the computer rooms at the three 
sites that are to use the technology curriculum are very well equipped. 
The only area of concern with regard to work space is the “war room” 
in the basement of the College of Education. Here conditions are 
somewhat cramped and communications are difficult. This problem 
will only be exacerbated if additional project personnel are to work in 
this area. 

Suggestion 1 

Recruitment of suitable personnel as worksite instructors has 
proved difficult, and various strategies have been tried. Since 
instructor recruitment may prove a problem for other programs, the 
team should document the variety of recruitment strategies employed. 

Su ggestion 2 

The technology- based curriculum may need to be reduced in scope 
because of the large amounts of time required for its development: 
several hundred hours for each hour of finished curriculum. The time 
factor is particularly relevant in view of the fact that Dr Powell’s 
allocation of time to the project will be reduced in Year 3. Therefore, 
it may be wise for the emphasis here to be very much on a multi-media 
approach that includes relatively few computer-based lesson units 
within a wider framework of print materials. For example, 20 hours of 
instruction might include 5 hours of computer-based learning 
modules. Even so, those 5 hours will need something like 1000-1500 
hours of development time — which is 150-200 person days. However, 
the amount of technology- based curriculum that can be produced will 
be easier to determine once work starts on the first modules. The 
team should also consider establishing some “drop-dead” dates for 
this portion of the project: if a suitable subject for technology 
development has not been selected by XX date, then the limited 
resources require goals to be reduced to XX level. Coming to a new 
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understanding about feasible goals and action plans for the technology- 
based curriculum should be a very high priority. 

Su ggestion 3 

With regard to work space, a serious concern is the basement “war 
room”, which is seen as barely adequate at present and which will 
become very crowded if more project personnel are working there in 
future. Given the key role of service provision and curriculum 
development in this project, some reallocation of space seems in 
order. Is there any possibility of obtaining additional or different space 
for the project to accommodate those working in this area? 

Goal 2: To conduct project evaluation and program impact research. 

The first year of the project has concentrated on workplace liaison 
and curriculum development, and so little has been done in the way of 
evaluation. Some assessment instruments for Davis Tool have been 
developed and a database has been set up to contain all project data 
and documentation. This will facilitate later impact research. 

However, it appears from interviews that the project team as a 
whole does not see evaluation as a high priority. This could affect 
project efficiency. Teachers and curriculum developers who share 
evaluation as a goal are often able to think of useful assessment 
approaches, gather corroborating evidence, and spot developing 
evaluation problems. When these people do not see impact evaluation 
as a major goal, careful data collection sometimes suffers and projects 
do not benefit from the timely evaluation insights of those closest to 
service delivery. 

Su ggestion 4 

Simul tan eous development of curriculum and impact assessments 
should be occurring. When curriculum developers do needs 
assessments, they can gather information about base-line data for 
productivity and organizational goals at the same time. When 
discussions for class recruitment occur, so can discussions for setting 
up the control groups indicated in the project proposal. 
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Goal 3: To disseminate the program, program products, and 

research findings. 

The project team has given this limited attention so far. Getting a 
cohesive team in place, solidifying workplace contacts and developing 
curriculum has demanded higher priority. One conference 
presentation arising from the project has been delivered. The project 
team as a whole seems to be in the initial stages of considering the 
goal of producing and disseminating a replicable model. It can be 
argued that dissemination responsibilities fall mainly to the Principal 
Investigators. If, however, all personnel are aware of the twin goals of 
service provision and producing a replicable model, then 
documentation of the model and lessons learned become a part of 
daily practice. The importance of keeping track of key events and 
problems /solutions as they happen at various stages of program 
development was discussed during the evaluators’ visit. 

Su ggestion 5 

All of the project team is urged to document the methods and 
techniques used in workplace liaison, needs assessment, curriculum 
development, teacher recruitment, instructional delivery, and learner 
assessment. The core of this already exists in some of the flow-charts 
and tools which have been developed. However, before these would be 
useful in disseminating a model, the charts would require descriptions 
of their use which would be essential for any other program developer 
who wanted to follow the model being piloted here. In addition, 
explanations of lessons learned in developing these tools can make 
them a good deal more valuable to other program developers. 
Accomplishing this goal involves more communication among project 
personnel about documenting the model, for example, sharing draft 
descriptions of processes and exchanging ideas on both the content of 
such descriptions and ways of laying them out clearly and consistently 
across all areas. 
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Goal 4: To promote program Institutionalization and diffusion. 

Although this goal will be addressed more toward the end of the 
project, when responsibility for the program will devolve to the 
workplace participants, documenting methodologies now will assist 
this future activity just as for the dissemination goal discussed above. 



In summary. Project ALERT has put together a strong team, which 
is now forging ahead very purposefully with its tasks. There are a few 
areas of concern to the evaluators, which are discussed above, but the 
future of the project appears to be in good hands and this should 
result in the innovative workplace literacy program described in the 
proposal. 
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Appendix — Project Documents Received 



Proposals 

Original grant proposal 
Year 2 continuation proposal 
Revised Year 3 proposal 

Resumes for project personnel 

Dr Dale Brandenburg, Co-Principal Investigator 

Dr Gary Powell, Co-Principal Investigator 

Dr Rita Richey, Co-Principal Investigator 

Dr Hal Stack, Co-Principal Investigator 

Ms Nancy Ruetz, Assistant Project Director of Curriculum 

Ms Chauncey Cooper, Workplace Instructor 

Human Subjects Review 

Project meeting minutes 

14 meetings between February and August 1995 

Time spent by project team 
Project tracking sheets 
Report on allocation of team members’ time 

Needs assessment 
Flow chart 

Data collection matrix 
Hourly employee interview 
Supervisor interview 
Observation checklist 
Data-base sample output 

Information about workplace sites 
Davis Tool description 
Detroit Axle description 
UAW-Chrysler Course Summary 
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Curriculum materials 

Whole language curriculum: 

complete set of lessons for Test-taking 

about half of set of lessons for Reading and for Math 

assessment measures for Test-taking 

Technology curriculum: 
overview plan 
software evaluation forms 

proposal for conference paper (Computer-based 
simulations for adult literacy) 

Formative interviews with project personnel 

Dr Dale Brandenburg, Co-Principal Investigator 

Dr Gary Powell. Co-Principal Investigator 

Dr Rita Richey, Co-Principal Investigator 

Dr Hal Stack, Co-Principal Investigator 

Ms Nancy Ruetz, Assistant Project Director of Curriculum 

Ms Maiy Jarvis. Graduate Research Assistant (Reading) 

Ms Irene Sinclair. Program Coordinator 

Ms Sheila Reed, Project Secretary 

Mr Steve Nelson. Davis Tool Human Resources 

Mr Mike Wysocki, Davis Tool Union Chairman 
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